benja-matic

`TypeError: forward() got an unexpected keyword argument 'cache_position'`

5

Hi, I'm seeing the error above when running `python -u run_text_generation.py --model_arch llama --model_name huggyllama/llama-13b --recent_ratio 0.1 --heavy_ratio 0.1 \`. Please let me know if you have a fix, as...

rename RotaryEmbedding -> GPTNeoXRotaryEmbedding in utils_hh/modify_gptneox.py This fixes the error `ImportError: cannot import name 'RotaryEmbedding' from 'transformers.models.gpt_neox.modeling_gpt_neox'` encountered in `run_text_generation.py`. This also updates the readme install instructions to specify transformers...

ModuleNotFoundError: No module named 'streaming_llm'

1

Hi, I'm seeing the above error when running `bash scripts/streaming/eval.sh h2o` after following the install instructions. If there is something else I need to install, could you point me to...

How to save and load ipex optimized model?

12

### Describe the issue Hi IPEX team, I have an application where I want to serve multiple models concurrently, and I want to share weights across concurrent instances. I normally...