OFQ

OFQ copied to clipboard

The +0.5 term in StatQ results in asymmetric weight quantization.

1

Unlike conventional symmetric weight quantization, I found that StatQ will let rounded integer weight add a term (+ 0.5), This item may cause the StatQ not to be actually deployed...

LuletterSoul

Hi, I have a quick question about the StatsQuantizer_specific_4_qkreparam_cga implementation. https://github.com/nbasyl/OFQ/blob/7ed37d1dd33d39395edbf49fcbbc52f678ecf961/src/quantization/quantizer/statsq.py#L191C1-L192C1 The final line of the forward pass is: quan_weights = quan_weights_no_grad.detach() - real_weights.detach() + real_weights but, (in if self.training)...

jiho264