AutoAWQ icon indicating copy to clipboard operation
AutoAWQ copied to clipboard

Any chance at supporting Nemotron?

Open ambroser53 opened this issue 1 year ago • 0 comments

I made minor some adjustments to the code to try and quantize Minitron-4B-Base (nemotron architecture has no gate_proj in the MLP) but the resulting model is completely unusable. I think more work has to go into it being supported but I do not know how to do that,

ambroser53 avatar Dec 19 '24 18:12 ambroser53