GrailFinder

is it possible to prune gptq models?

3

``` def get_llm(model, cache_dir="llm_weights"): model = AutoModelForCausalLM.from_pretrained( model, torch_dtype=torch.float16, cache_dir=cache_dir, low_cpu_mem_usage=True, device_map="auto" ) model.seqlen = 2048 return model ``` I am interested if it is possible to prune 4-bit gptq...