theScotchGame issues

Repositories
Issues
Comments

Results 1 issues of


                                            theScotchGame

GPU Usage Keeps High Even Without Inference Load

Configuration: AMD W7900 + Rocm5.6 ![image](https://github.com/turboderp/exllama/assets/36061851/8e7639ec-8938-4991-bc97-3c0ce3f92638) Running the model on oobabooga/text-generation-webui, GPU Usage keeps even unload the model. Model: TheBloke/Llama-2-70B-GPTQ:gptq-4bit-32g-actorder_True Running meta-llama/Llama-2-7b-chat-hf without quantiziation would not have this issue. Is...