[Bug] Cannot use Tritonserver with MMdeploy custom operations

[firatbey]

Checklist

• [X] I have searched related issues but cannot get the expected help.
• [X] 2. I have read the FAQ documentation but cannot get the expected help.
• [x] 3. The bug has not been fixed in the latest version.

Describe the bug

I tried to use mmdeploy custom operations in Triton server docker container. Goal is to run rotated_rtmdet_m-3x-dota onnx model with custom operation "NMSRotated".

I created a docker image that pulls libraries from mmdeploy and adds to triton server image

The docker container start, does not produce any errors but does not load any models.

Reproduction

Docker file to build triton image with mmdeploy custom operations.

FROM openmmlab/mmdeploy:ubuntu20.04-cuda11.8-mmdeploy1.3.1
FROM nvcr.io/nvidia/tritonserver:22.04-py3
COPY --from=openmmlab/mmdeploy:ubuntu20.04-cuda11.8-mmdeploy1.3.1 /root/workspace/mmdeploy/build/lib/* /opt/tritonserver/lib/
RUN wget https://github.com/microsoft/onnxruntime/releases/download/v1.15.1/onnxruntime-linux-x64-1.15.1.tgz \
	&& tar -zxvf onnxruntime-linux-x64-1.15.1.tgz
ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/tritonserver/onnxruntime-linux-x64-1.15.1/lib" 

Note: I needed to add onnxruntime-linux-x64-1.15.1 otherwise I get error: Result: cannot load "libmmdeploy_onnxruntime_ops.so"

Execution command for running docker container:

docker run --gpus=1 --rm -p 8000:8000 -p 8001:8001 -p 8002:8002 -v /home/user/model:/models --env LD_PRELOAD=/opt/tritonserver/lib/libmmdeploy_onnxruntime_ops.so mmdeploy-triton tritonserver --model-repository=/models

Environment

FROM openmmlab/mmdeploy:ubuntu20.04-cuda11.8-mmdeploy1.3.1
FROM nvcr.io/nvidia/tritonserver:22.04-py3
COPY --from=openmmlab/mmdeploy:ubuntu20.04-cuda11.8-mmdeploy1.3.1 /root/workspace/mmdeploy/build/lib/* /opt/tritonserver/lib/
RUN wget https://github.com/microsoft/onnxruntime/releases/download/v1.15.1/onnxruntime-linux-x64-1.15.1.tgz \
	&& tar -zxvf onnxruntime-linux-x64-1.15.1.tgz
ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/opt/tritonserver/onnxruntime-linux-x64-1.15.1/lib" 

### Error traceback

```Shell
=============================
== Triton Inference Server ==
=============================

NVIDIA Release 22.04 (build 36821869)
Triton Server Version 2.21.0

Copyright (c) 2018-2022, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

I0708 08:47:12.501680 1 libtorch.cc:1381] TRITONBACKEND_Initialize: pytorch
I0708 08:47:12.501761 1 libtorch.cc:1391] Triton TRITONBACKEND API version: 1.9
I0708 08:47:12.501765 1 libtorch.cc:1397] 'pytorch' TRITONBACKEND API version: 1.9
2024-07-08 08:47:12.605144: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
I0708 08:47:12.636185 1 tensorflow.cc:2181] TRITONBACKEND_Initialize: tensorflow
I0708 08:47:12.636211 1 tensorflow.cc:2191] Triton TRITONBACKEND API version: 1.9
I0708 08:47:12.636215 1 tensorflow.cc:2197] 'tensorflow' TRITONBACKEND API version: 1.9
I0708 08:47:12.636218 1 tensorflow.cc:2221] backend configuration:
{}
I0708 08:47:12.637640 1 onnxruntime.cc:2400] TRITONBACKEND_Initialize: onnxruntime
I0708 08:47:12.637653 1 onnxruntime.cc:2410] Triton TRITONBACKEND API version: 1.9
I0708 08:47:12.637657 1 onnxruntime.cc:2416] 'onnxruntime' TRITONBACKEND API version: 1.9
I0708 08:47:12.637660 1 onnxruntime.cc:2446] backend configuration:
{}
I0708 08:47:12.658508 1 openvino.cc:1207] TRITONBACKEND_Initialize: openvino
I0708 08:47:12.658518 1 openvino.cc:1217] Triton TRITONBACKEND API version: 1.9
I0708 08:47:12.658522 1 openvino.cc:1223] 'openvino' TRITONBACKEND API version: 1.9
I0708 08:47:14.017421 1 pinned_memory_manager.cc:240] Pinned memory pool is created at '0x7fc320000000' with size 268435456
I0708 08:47:14.020986 1 cuda_memory_manager.cc:105] CUDA memory pool is created on device 0 with size 67108864
I0708 08:47:14.028413 1 model_repository_manager.cc:1077] loading: rotated_rtmdet_m-3x-dota:1
I0708 08:47:14.128733 1 onnxruntime.cc:2481] TRITONBACKEND_ModelInitialize: rotated_rtmdet_m-3x-dota (version 1)
I0708 08:47:14.132534 1 onnxruntime.cc:2524] TRITONBACKEND_ModelInstanceInitialize: rotated_rtmdet_m-3x-dota (GPU device 0)