❓ [Question] How to install torch_tensorrt python API in ubuntu 20.04?

[IamExperimenting]

❓ Question

I want to install torch_tensorrt python API in ubuntu 20.04. could you please provide step by a step installation procedure? I tried pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

when I try to import the module import torch_tensorrt

I'm getting the below error,

Environment

Build information about Torch-TensorRT can be found by turning on debug messages

• PyTorch Version (e.g., 1.0): 1.11.0
• CPU Architecture:
• OS (e.g., Linux): LINUX
• How you installed PyTorch (conda, pip, libtorch, source): conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
• Build command you used (if compiling from source):
• Are you using local sources or building from archives:no
• Python version: 3.7.13
• CUDA version: 11.3.1
• GPU models and configuration:
• Any other relevant information:

@narendasan @peri044

[dignakov]

Looks like it can't find one of it's dependencies.

You could try following the PyTorch docs: https://pytorch.org/TensorRT/tutorials/installation.html

You'll need to install the different parts manually to get the wheel to eventually work, it does not bundle anything.

(1) install CUDA: https://developer.nvidia.com/cuda-zone (2) install CUDNN: https://developer.nvidia.com/cudnn (3) install TensorRT: https://developer.nvidia.com/tensorrt

(in a venv) (4) install PyTorch: https://pytorch.org/ (5) pip install tensorrt pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

If you run into missing instructions you can post here and I'll try to go back through my bash history.

This is what I do: (1)

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin

sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"

sudo apt update

sudo apt install cuda

(2) and (3) Get the debs from the links in PyTorch docs and install.

(in a virtual env) (4) I use pip, so I pip install pytorch (5) pip install tensorrt pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

[IamExperimenting]

@dignakov , thanks for your reply,

Steps i followed the below steps,

1. https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-pip - i followed this step to install tensorrt
2. then using pip command pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases I installed torch_tensorrt. however, the weird part is, if import torch_tensorrt as a first module in script it throws the same error

Error

same as my above screenshot.

if i do like the below steps

import torch
import tenosrrt
import torch_tensorrt

it gives warning.

[dignakov]

It doesn't look like those instructions are complete as you're still missing libnvinfer. So I would follow what's in the PyTorch docs.

I've only been able to get a successful system up and running using what I posted . . . Other instructions may work, but I find they're sometimes missing steps or assume you've done something without being explicit about it.

If you're confident you've got cuda, cudnn, and tensorrt running, the other thing to check is if you've got proprietary nvidia drivers installed and they can see your gpu.

(1) Can you run (in a terminal) nvidia-smi and post what you get.

(2) In python, check if it can see your GPU:

import torch
torch.cuda.is_available()  # should return True
torch.cuda.device_count()  # how many devices you have
torch.cuda.current_device()  # which one you're using now
torch.cuda.get_device_name(0)  # what you got above to see what it thinks it is

[IamExperimenting]

@dignakov you're right it didn't work now after I restarted the kernel.

could you please help me to install it? To be honest, I'm familiar with installing using Linux commands. But I'm trying my best.

[dignakov]

@IamExperimenting can you try following what I had in my comment above (https://github.com/pytorch/TensorRT/issues/1133#issuecomment-1159780655)?

(1) is exactly how I usually do it.

for (2) and (3) you can just download the debs and run sudo apt install ./whatever.deb

I don't use conda, so if you want to follow the instructions below, you need to deactivate any conda environments.

(4) and (5) - I make a virtual environment

python3 -m venv env

source it source env/bin/activate (you'll need to do this every time you want to use pytorch)

install pytorch pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu113

install torch-tensorrt pip3 install torch-tensorrt -f https://github.com/NVIDIA/Torch-TensorRT/releases

[IamExperimenting]

@dignakov , after some continuous debugging and with your support. Now, I'm able install torch_tensorrt. please find the screenshot

however, I still have a question, when I looked into PyTorch official installation page, it has many steps like installing C++ and bazel and python3 setup.py [install/bdist_wheel] and python3 setup.py install --use-cxx11-abi

So, here my question is do I still need to install all those or am I good to go with the above installation?

https://pytorch.org/TensorRT/tutorials/installation.html

I'm still clear with the instruction document to install torch_tensorrt.

it would really help me and others to understand? @dignakov if possible could you please put down a step by step to install for python API.

[dignakov]

Unless you'd like to build everything from source, you should be okay just with the "Precompiled Binaries" section. That's everything you need for the "Python API", if you can import torch_tensorrt and use it, you shouldn't need to do anything else.

The section you're talking about is titled "Compiling From Source", and that's for building everything from scratch . . .

[IamExperimenting]

@dignakov thanks very much for your support.

[IamExperimenting]

@dignakov I'm still facing the error ERROR: [Torch-TensorRT] - Cannot get current device

[IamExperimenting]

I'm still facing an error, I followed the instruction you provided earlier.

as a first step :

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin

sudo mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600

sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub

sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"

sudo apt update

sudo apt install cuda

Step: 2 I download the CUDNN from the link which you provided https://developer.nvidia.com/rdp/cudnn-download name of the .deb file cudnn-local-repo-ubuntu2004-8.4.1.50_1.0-1_amd64.deb then I ran the below command sudo dpkg -i cudnn-local-repo-ubuntu2004-8.4.1.50_1.0-1_amd64.deb after running that I get the below error.

[narendasan]

Are you trying to setup cuda or cudnn?

[IamExperimenting]

@narendasan I'm trying to setup CUDA, CUDNN and TensorRT. Finally, to install torch_tensorrt.

[github-actions[bot]]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[narendasan]

Can you try installing the latest package, as long as you have cuda installed all other dependencies will get installed for you via pip. If the issue persists reopen the issue.