ONNXRuntime-GPU Fails in ComfyUI – Possible cuDNN Conflict with CTranslate2

[xuxinda]

Environment

• platform: Linux-6.8.0-53-generic-x86_64-with-glibc2.39
• CUDA: 12.4
• Driver: 550.144.03
• Python 3.12.3

Key Packages

• torch 2.6.0+cu124
• ctranslate2 4.5.0
• onnxruntime-gpu 1.22.0
• nvidia-cublas-cu12 12.4.5.8
• nvidia-cuda-cupti-cu12 12.4.127
• nvidia-cuda-nvrtc-cu12 12.4.127
• nvidia-cuda-runtime-cu12 12.4.127
• nvidia-cudnn-cu12 9.1.0.70
• nvidia-cufft-cu12 11.2.1.3
• nvidia-curand-cu12 10.3.5.147
• nvidia-cusolver-cu12 11.6.1.9
• nvidia-cusparse-cu12 12.3.1.170
• nvidia-cusparselt-cu12 0.6.2
• nvidia-ml-py 12.570.86
• nvidia-nccl-cu12 2.21.5
• nvidia-nvjitlink-cu12 12.4.127
• nvidia-nvtx-cu12 12.4.127

Error (When running ComfyUI with ONNXRuntime-GPU)

[E:onnxruntime:, inference_session.cc:2105 operator()] Exception during initialization: /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:123 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudnnStatus_t; bool THRW = true; std::conditional_t<THRW, void, common::Status> = void] /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:116 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudnnStatus_t; bool THRW = true; std::conditional_t<THRW, void, common::Status> = void] CUDNN failure 4: cuDNN could not be found or could not be loaded. ; GPU=0 ; hostname=gpu8 ; file=/onnxruntime_src/onnxruntime/core/providers/cuda/cuda_execution_provider.cc ; line=181 ; expr=cudnnCreate(&cudnn_handle_);
!!! Exception during processing !!! [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Exception during initialization: /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:123 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudnnStatus_t; bool THRW = true; std::conditional_t<THRW, void, common::Status> = void] /onnxruntime_src/onnxruntime/core/providers/cuda/cuda_call.cc:116 std::conditional_t<THRW, void, onnxruntime::common::Status> onnxruntime::CudaCall(ERRTYPE, const char*, const char*, ERRTYPE, const char*, const char*, int) [with ERRTYPE = cudnnStatus_t; bool THRW = true; std::conditional_t<THRW, void, common::Status> = void] CUDNN failure 4: cuDNN could not be found or could not be loaded. ; GPU=0 ; hostname=gpu8 ; file=/onnxruntime_src/onnxruntime/core/providers/cuda/cuda_execution_provider.cc ; line=181 ; expr=cudnnCreate(&cudnn_handle_);

Suspected Cause

In the same Python environment, this error does not occur when running code directly — it only happens inside ComfyUI.

ComfyUI loads cuDNN from CTranslate2 (libcudnn-74a4c495.so.9.1.0) first, which may conflict with ONNXRuntime-GPU. When I use:

export LD_PRELOAD=env/lib/python3.12/site-packages/nvidia/cudnn/lib/libcudnn.so.9

The error disappears.

Question

Is there a better way to avoid this cuDNN conflict without using LD_PRELOAD every time? Any clean solution to make ONNXRuntime-GPU and CTranslate2 coexist in the same Python environment?