❓ [Question] Not enough inputs provided (runtime.RunCudaEngine)

[Pekary]

❓ Question

i make a pressure test on my model compiled by torch-tensorrt, it will report errors after 5 minutes, the traceback as blow:

2022-09-09T09:16:01.618971735Z   File "/component/text_detector.py", line 135, in __call__
2022-09-09T09:16:01.618975181Z     outputs = self.net(inp)
2022-09-09T09:16:01.618978313Z   File "/miniconda/envs/python36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
2022-09-09T09:16:01.618981965Z     return forward_call(*input, **kwargs)
2022-09-09T09:16:01.618985142Z RuntimeError: The following operation failed in the TorchScript interpreter.
2022-09-09T09:16:01.618988457Z Traceback of TorchScript, serialized code (most recent call last):
2022-09-09T09:16:01.618991980Z   File "code/__torch__.py", line 8, in forward
2022-09-09T09:16:01.618995305Z     input_0: Tensor) -> Tensor:
2022-09-09T09:16:01.618998495Z     __torch___ModelWrapper_trt_engine_ = self_1.__torch___ModelWrapper_trt_engine_
2022-09-09T09:16:01.619001820Z     _0 = ops.tensorrt.execute_engine([input_0], __torch___ModelWrapper_trt_engine_)
2022-09-09T09:16:01.619005168Z          ~~~~~~~~~~~~~~~~~~~~~~~~~~~ <--- HERE
2022-09-09T09:16:01.619008442Z     _1, = _0
2022-09-09T09:16:01.619011485Z     return _1
2022-09-09T09:16:01.619014563Z 
2022-09-09T09:16:01.619017565Z Traceback of TorchScript, original code (most recent call last):
2022-09-09T09:16:01.619020865Z RuntimeError: [Error thrown at core/runtime/register_trt_op.cpp:101] Expected compiled_engine->exec_ctx->allInputDimensionsSpecified() to be true but got false
2022-09-09T09:16:01.619024625Z Not enough inputs provided (runtime.RunCudaEngine)

then i get an error about cuda memory illegal access:

2022-09-13T02:32:46.621963863Z   File "/component/text_detector.py", line 136, in __call__
2022-09-13T02:32:46.621966267Z     inp = inp.cuda()
2022-09-13T02:32:46.621968419Z RuntimeError: CUDA error: an illegal memory access was encountered

What you have already tried

I have tried upgrade the pytorch version from 1.10.0 to 1.10.2, also tried upgrade torch to 1.11.0 python 3.7, but it didn't works.

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version (e.g., 1.0): 1.10.2
• CPU Architecture: x86
• OS (e.g., Linux): centos 7
• How you installed PyTorch (conda, pip, libtorch, source): pip
• Build command you used (if compiling from source): /
• Are you using local sources or building from archives: no
• Python version: 3.6
• CUDA version: 11.3
• GPU models and configuration: gpu is nvidia-T4 with 16G memory
• Any other relevant information:

Additional context