[RuntimeError] <CUDA error: an illegal memory access was encountered>

[Dhaizei]

Is your feature request related to a problem? Please describe.

RuntimeError: CUDA error: an illegal memory access was encountered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Solutions

我仅仅是 让这个模型一直运行，没有存储历史数据，用了两张卡来运行的。怎么解决？

Additional context

No response

[leoluopy]

mark , same problem , have you find a solution ? @Dhaizei

[Dhaizei]

i give up. you can try fastchat to use this model