pangr

PTQ is faster than QAT

2

## Description ## Environment **TensorRT Version**: 8.4.1.5 **NVIDIA GPU**: 1080ti **NVIDIA Driver Version**: 450 **CUDA Version**: 11.0 **CUDNN Version**: 8.1.0 **Operating System**: **Python Version (if applicable)**: **Tensorflow Version (if applicable)**:...

what does “ConstShuffleFusion” mean

1

## Description ## Environment **TensorRT Version**: **NVIDIA GPU**: **NVIDIA Driver Version**: **CUDA Version**: **CUDNN Version**: **Operating System**: **Python Version (if applicable)**: **Tensorflow Version (if applicable)**: **PyTorch Version (if applicable)**: **Baremetal...

PRelu is not integrated with QDQ

1

## Description ## Environment **TensorRT Version 8.4**: **NVIDIA GPU 1080ti**: **NVIDIA Driver Version 470**: **CUDA Version 11.0**: **CUDNN Version**: **Operating System**: **Python Version (if applicable)**: **Tensorflow Version (if applicable)**: **PyTorch...

How to add QDQ before LeakyRelu

1

## Description ## Environment **TensorRT Version 8.4: **NVIDIA GPU p40: **NVIDIA Driver Version 470: **CUDA Version 11.0: **CUDNN Version 8.1: **Operating System linux: **Python Version (if applicable) 3.6: **Tensorflow Version...

What does “Reformatting CopyNode for Input Tensor” mean in trtexec' dump profile

4

## Description What does “Reformatting CopyNode for Input Tensor” mean in trtexec' dump profile ## Environment **TensorRT Version**: **NVIDIA GPU**: **NVIDIA Driver Version**: **CUDA Version**: **CUDNN Version**: **Operating System**: **Python...

[BUG]After using the code that supports llama inference, the result of the inference is different from the original one

2

**Describe the bug** A clear and concise description of what the bug is. **To Reproduce** Steps to reproduce the behavior: 1. Simple inference script to reproduce 2. What packages are...

[BUG]After using the code that supports llama inference, the result of the inference is different from the original one

2

When I asked "Who is founder of goolge.com?", the result of llama13B answered as shown in the figure below: “tro tro tro tro tro tro tro tro tro tro tro...

**Describe the bug** The results generated based on DS inference and the results generated by the original model have certain errors, and the results generated by DS inference will be...

An error is reported after executing mlc_chat_cli

1

 下面是vulkaninfo --summary的结果