Jiayi Yuan

icon indicating a website link jy-yuan.github.io

icon location USA icon location CS PhD student @ Rice University, BS CS @ Tsinghua University

Results 4 comments of Jiayi Yuan

not support evaluation with ROCM

Sure! We welcome contributions from the research community to improve KIVI.

Spport for ChatGLM3

Thanks for your interest! You can modifiy the `modeling_chatglm.py` in huggingface similar to our implementation in [llama_kivi.py](https://github.com/jy-yuan/KIVI/blob/main/models/llama_kivi.py) or [mistral_kivi.py](https://github.com/jy-yuan/KIVI/blob/main/models/mistral_kivi.py). We are planning to extend KIVI to more models and benchmark...

Where is the falcon_kivi?

Hi, @Felixvillas; thanks for your patience; we have updated the corresponding mistral and falcon files to the branch lmeval.

Unable to Reproduce Results for LongBench

Hi, For Llama-3 Instruct models, please add the prompt template as shown [here](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct). We've updated the code in pred_long_bench.py accordingly. Please give it a try, and feel free to ask...