Supported Models

Open TechxGenus opened this issue 2 years ago • 1 comments

Congratulations on coming up with such an excellent quantization algorithm! I'm trying to use AQLM to quantize Deepseek-Coder and Starcoder2, but the repository doesn't seem to have direct support. Are there any plans to support more models? Or any suggestions on how to modify the source code to achieve fast support, thanks.

Mar 10 '24 10:03 TechxGenus

Hi! Only a minor modification is needed to be able to run a different model architecture. Hopefully, a change like this should be enough to run starcoder2. You can try it out in the starcoder2 branch.

Mar 10 '24 21:03 BlackSamorez

Thanks!

Mar 11 '24 03:03 TechxGenus