[BUG in Stable Diffusion inference] There's an error on CUDAGraph when using deepspeed inference. How to fix it?

[foin6]

【Repo Link】: Stable-Diffusion inference

【The Command used to run】: deepspeed --num_gpus 1 test-stable-diffusion.py

【Envs】:

RTX 3090
deepspeed 0.12.6
torch 1.13.1
diffusers 0.26.1
triton 2.0.0.dev20221202

【Traceback information】:

It seems like the CUDAGraph is not available, which confused me a lot. Please take time to review the issue and give me some feedback. Appreciate!

[foin6]

Would be nice if someone can tell how to fix it.

[umchand]

Can you try upgrading the torch to 2.2.1 and use the latest deepspeed ?