david0963658

I encountered the same issue in Python3.11.  When I switched my library version to the following, the issue was resolved. ``` accelerate==0.27.2 aiohttp==3.9.3 aiosignal==1.3.1 anyio==4.3.0 attrs==23.2.0 auto_gptq==0.7.1 certifi==2024.2.2 charset-normalizer==3.3.2...

By setting temperature=0.7 in the model.generate function, the error resolved. `generated_ids = model.generate( **model_inputs, max_new_tokens=512, temperature=0.7 )`