LIU Man comments

Repositories
Issues
Comments

Results 4 comments of


                                            LIU Man

How to convert model trained with QuantNoise to quantized model runnable on CPU only?

Hi @stolam I have also tried iPQ, but got the same problem that the quantized model cannot run only on CPU. I also tried torch dynamic_quantization, but encounter a lot...

terminal_predict.py在线预测出错

同样的问题，也是匹配不上

Problem in the generation with a transformer model trained with Iterative Product Quantization

I'am troubled by it, too lol

KV Cache in the transformers

It seems yes. Please check this: https://github.com/facebookresearch/fairseq/blob/920a548ca770fb1a951f7f4289b4d3a0c1bc226f/fairseq/model_parallel/modules/multihead_attention.py#L128