gloritygithub11

Hi @Barry-Delaney , any updates on this?

Thanks @byshiue for the response. Will it be supported at sometime in future?

@byshiue I didn't use docker for dev. docker/common/install_tensorrt.sh is executed in my local env, and tensorrt has already been upgraded to 10.0.1

I tried rebuild with docker image, get the same error

@byshiue yes, I've rebuild the docker image. You can see in my above list, the tensorrt already 0.10.1: libnvinfer_plugin_tensorrt_llm.so.10.0.1 ``` ll /usr/local/tensorrt/lib/ total 3.5G lrwxrwxrwx 1 root root 20 Apr...

with following docker file ``` # Use an official NVIDIA CUDA image as a parent image FROM nvidia/cuda:12.4.1-devel-ubuntu20.04 # Set the working directory WORKDIR /app # Install software-properties-common to add...

@byshiue I follow the steps in https://nvidia.github.io/TensorRT-LLM/installation/linux.html to create a new docker env. Get similar error. ``` Process 0 loading engine from /root/models/tmp/trt_engines/Meta-Llama-3-8B-Instruct/fp16/1-gpu-tp1/rank0.engine [05/24/2024-08:20:11] [TRT] [I] Loaded engine size: 15323...

I build a new docker image with: ``` make release_build CUDA_ARCHS="80-real" ``` the image could build and I can use this image to convert & build with following command: ```...

``` $ cat /usr/local/tensorrt/include/NvInferVersion.h | grep version //! Defines the TensorRT version #define NV_TENSORRT_MAJOR 10 //!< TensorRT major version. #define NV_TENSORRT_MINOR 0 //!< TensorRT minor version. #define NV_TENSORRT_PATCH 1 //!<...

do you mean `/app/tensorrt_llm/`, looks there's no python related content in the folder ``` ll /app/tensorrt_llm/ total 12 drwxr-xr-x 1 root root 40 May 29 13:01 ./ drwxr-xr-x 1 root...