cnahmgx

查了一些资料，说是要修改自定义词库的使用权重，请问下如何修改

CUDA Setup failed despite GPU being available. Inspect the CUDA SETUP outputs above to fix your environment!

9

===================================BUG REPORT=================================== Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues ================================================================================ CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so CUDA SETUP: Highest compute capability among GPUs...

import flash_attn rotary fail

2

背景：在本地部署ModelScope-Agent-7B，机器为nvidia的A100，速度特别慢，chat一次平均耗时18秒 已经按照[https://modelscope.cn/models/iic/ModelScope-Agent-7B/summary的步骤安装了flash-attention==2.3.5、layer_norm、rotary-embedding-torch==0.5.3] 启动ModelScope-Agent-7B 还是报：Warning: import flash_attn rotary fail, please install FlashAttention rotary to get higher efficiency https://github.com/Dao-AILab/flash-attention/tree/main/csrc/rotary