open-genie icon indicating copy to clipboard operation
open-genie copied to clipboard
verified publisher icon myscience

Entropy loss bug in Lookup Free Quantization

Open flm8620 opened this issue 6 months ago • 0 comments

https://github.com/myscience/open-genie/blob/732b9f9b746f18fff1a0fb22f83638224f2f7cc6/genie/module/quantization.py#L125

entropy_loss = inp_ent + self.diversity_weight * avg_ent

This line should be

entropy_loss = inp_ent - self.diversity_weight * avg_ent

flm8620 avatar Aug 17 '25 08:08 flm8620