GongTianxiang comments

Repositories
Issues
Comments

Results 2 comments of


                                            GongTianxiang

[Bug] 超长消息列表性能优化

我是用vercel一键部署的，在windows11 edge浏览器上使用。本地聊天框对话超过100 就会出现卡顿的现象了。但是只有输入卡顿，输出速度还是正常的。希望能尽快优化这个问题。

RuntimeError: CUDA error: device-side assert triggered when running Llama on multiple gpus

I also have the same problem. I can use the vicuna 13b model properly with --load-8bit option on in single 4090 GPU, but when I use multiple gpus like (--num-gpus...