Heng Guo
Heng Guo
retest
[retest](https://inteltf-jenk.sh.intel.com/job/intel-lpot-validation-top-mr-extension/3950/)
skx-8180 batch_size: 100 resnet50: ori: 0.63272 qat: 0.61474 resnet101: ori: 0.63438 qat: 0.54072
[retest](https://inteltf-jenk.sh.intel.com/job/intel-lpot-validation-top-mr-extension/4007/artifact/report.html)
pre-commit.ci autofix
> Could we add a document introducing what h2o is? add in the example/readme
> Hi [@wenhuach21](https://github.com/wenhuach21) [@n1ck-guo](https://github.com/n1ck-guo), does export for q4_k work right now? I tried to adapt [that](https://github.com/intel/auto-round/blob/9a6f325adf3724537271891ae9240f18e5612382/auto_round/export/export_to_gguf/convert.py#L1145-L1188) for torchao, and tried to serve with vllm `vllm serve ./phi4-mini-torchao-ar-gguf-q4_k-3.8B-Q4_K_S.gguf --tokenizer microsoft/Phi-4-mini-instruct --device...
We have tested the code of export q4_k_s. For some other models, it works well. But for microsoft/Phi-4-mini-instruct, we cannot export and it will raise error. This is because our...
@jerryzh168 Thank you for waiting. This issue seems to be caused by a problem with the llama.cpp version. Could you please try with this pr https://github.com/intel/auto-round/pull/524 and the lastest gguf-py....
https://github.com/intel/auto-round/pull/1043 https://github.com/intel/auto-round/pull/1031