BC6H MSE calculation error

[Zelex]

There appears to be an error when calculating MSE for BC6H, where with the following files

https://send.firefox.com/download/dcc3a03777702c22/#ARpj0hUbj52fHzQkqv96Tw

Compressonator is saying MSE is 1.6773 Where as the correct MSE should be 0.0414 from what I can tell from other programs.

[Zelex]

When comparing two different EXR files (no DDS involved to decode), the MSE is also calculated incorrectly (with the same incorrect result). Thus, perhaps this is an issue with float textures in general?

[NPCompress]

Hi @Zelex, can you provide a reference link to the "other programs" you used to determine the 0.0414 value. Thx.

[Zelex]

I'll make one for you, 1 sec.

[Zelex]

https://send.firefox.com/download/990dd959f0492c5d/#X0pnxs-4ZgpxYvlx-_Ssmg

[NPCompress]

@Zelex Thank you for the reference.