LLaMA3-Quantization icon indicating copy to clipboard operation
LLaMA3-Quantization copied to clipboard
verified publisher icon Macaronlin

How to eval based quant model?

Open xd1073321804 opened this issue 1 year ago • 0 comments

Hi，you have provided GPTQ int4 quant model in https://huggingface.co/LLMQ/LLaMA-3-8B-GPTQ-4bit-b128/tree/main, How to evaluate metric in your code? I have try run "sh scripts/eval_gptq_commonsenseqa.sh", but it only evaluate FP32 model metric，when remove '--quant_method gptq' in script, it meet error like this. error Can you provide solution for this question? Thanks.

xd1073321804 avatar May 06 '24 09:05 xd1073321804