IQA-PyTorch icon indicating copy to clipboard operation
IQA-PyTorch copied to clipboard

How to use PLCC metrics to evaluate algorithms whose outputs are lower the better the image quality is?

Open Zhang-Jing-Xuan opened this issue 2 years ago • 4 comments

Some evaluation algorithms, such as Brisque, have a lower result, the better the image quality, in this case how to calculate the PLCC to compare with those algorithms that have a higher result the better the image quality? If they use the same human eye score, then Brisque will always be negatively correlated with the human eye score, and the calculated PLCC will always be negative.

Zhang-Jing-Xuan avatar Jul 22 '23 02:07 Zhang-Jing-Xuan

You can simply use the absolute PLCC value if the metric is lower better.

chaofengc avatar Jul 22 '23 07:07 chaofengc

Thank you for your reply. Can I take the inverse of the results of algorithms (x->1/x) where the smaller the result the higher the quality and then calculate the PLCC?

Zhang-Jing-Xuan avatar Jul 22 '23 08:07 Zhang-Jing-Xuan

It is OK to do so, but I think not a good practice. Because PLCC measures the linear relationship, but 1/x is not a linear function. You can simply normalize the values with min and max value like $1 - (x - min)/(max - min)$

chaofengc avatar Jul 25 '23 03:07 chaofengc

OK. Thank you for your reply.

Zhang-Jing-Xuan avatar Jul 25 '23 11:07 Zhang-Jing-Xuan