AutoAWQ
AutoAWQ copied to clipboard
Any chance at supporting Nemotron?
I made minor some adjustments to the code to try and quantize Minitron-4B-Base (nemotron architecture has no gate_proj in the MLP) but the resulting model is completely unusable. I think more work has to go into it being supported but I do not know how to do that,