TensorRT_Inference_Demo
TensorRT_Inference_Demo copied to clipboard
Li-Hongda
执行可执行文件的时候报错了
root@rtx2060:/home/tuc/TensorRT_Inference_Demo-main/bin# ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images/bus.jpg
[08/16/2023-14:32:00] [I] [TRT] [MemUsageChange] Init CUDA: CPU +547, GPU +0, now: CPU 554, GPU 243 (MiB)
[08/16/2023-14:32:01] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +273, GPU +68, now: CPU 846, GPU 311 (MiB)
[08/16/2023-14:32:01] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-14:32:01] [I] [TRT] Input filename: ../weights/yolov5/yolov5s.onnx
[08/16/2023-14:32:01] [I] [TRT] ONNX IR version: 0.0.6
[08/16/2023-14:32:01] [I] [TRT] Opset version: 11
[08/16/2023-14:32:01] [I] [TRT] Producer name: pytorch
[08/16/2023-14:32:01] [I] [TRT] Producer version: 1.11.0
[08/16/2023-14:32:01] [I] [TRT] Domain:
[08/16/2023-14:32:01] [I] [TRT] Model version: 0
[08/16/2023-14:32:01] [I] [TRT] Doc string:
[08/16/2023-14:32:01] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-14:32:01] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/16/2023-14:32:01] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-14:32:01] [E] [TRT] 4: [network.cpp::operator()::3020] Error Code 4: Internal Error (images: kOPT dimensions in profile 0 are [8,3,640,640] but input has static dimensions [1,3,640,640].)
[08/16/2023-14:32:01] [E] [TRT] 2: [builder.cpp::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
object_detection: /home/tuc/TensorRT_Inference_Demo-main/src/basemodel.cpp:57: void Model::OnnxToTRTModel(): Assertion `data' failed.
Aborted (core dumped)
按照您的教程readme文档的说明，使用yolov5-v6.1 仓库，执行完
python export.py --weights=yolov5s.pt --dynamic --simplify --include=onnx --opset 11
命令得到了onnx模型，并将onnx模型放入到 configs 文件夹的同级目录下的 weights 文件夹下的yolov5文件夹中。
接着进入 bin 文件夹，执行
./object_detection yolov5 /home/tuc/yolov5-6.1/data/images/bus.jpg
命令，就出现了开头的core dump
请问 configs 下的 yolov5.yaml 文件中的 engine_file 字段的trt 文件是需要自己手动生成吗？
input has static dimensions [1,3,640,640] 报错的原因是ONNX文件为静态shape，可以使用netron检查一下，然后重新生成一下onnx。或者注释掉代码中关于dynamic shape的处理
我用trtexec工具生成了yolov5s.trt，并放在了weights下，然后执行 ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images 命令，显示了推理时间，却没有在保存的图片上将检测框画出来。 如下图
那就已经是float了，我刚刚测试了这两张图应该是没有问题的，因此我认为还是trt转换出了问题，你可以把config中的dynamic设置为0重新生成一下（因为从报错信息来看onnx为静态文件）
我把dynamic设为了0，还是不起作用。 您是怎么得到trt文件的，我是用这个命令得到的：
root@rtx2060:/usr/local/TensorRT-8.4.3.1/bin# ./trtexec --onnx=/home/tuc/yolov5-6.1/weights/yolov5s.onnx --saveEngine=/home/tuc/yolov5-6.1/weights/outfpbest.trt --workspace=2048 --best
# 导出onnx是用这个命令，yolov5版本是6.1
python export.py --weights=yolov5s.pt --dynamic --simplify --include=onnx --opset 11
会不会和环境有关 我用的是cuda 11.2 , cudnn 8.4.0 , TensorRT-8.4.3.1 rtx2060
那就已经是float了，我刚刚测试了这两张图应该是没有问题的，因此我认为还是trt转换出了问题，你可以把config中的dynamic设置为0重新生成一下（因为从报错信息来看onnx为静态文件）
之前的报错信息在我导出 trt 模型后，放在weights路径下就自动消失了。
我这个代码是如果weights路径下没有检测到相应的权重，就调用转换onnx转trt函数，之前的报错是发生在转换这一步；你用trt转好之后就不经过这一步了，所以不报错。但是从当时的报错信息能看出来，你所用的onnx是静态图的，所以在转换成trt时应该把dynamic设置为0，我的意思就是重新转换一下（先把weights下的engine删掉）。版本可能也有关吧，这个得查一下相关的更新文件，但应该不是主要原因。
很奇怪，我删除了原先的onnx 和 trt 文件
重新使用export.py导出了onnx，python export.py --weights=weights/yolov5s.pt --include=onnx
(tc_yolov5) root@rtx2060:/home/tuc/yolov5-6.1# python export.py --weights=weights/yolov5s.pt --include=onnx
export: data=data/coco128.yaml, weights=['weights/yolov5s.pt'], imgsz=[640, 640], batch_size=1, device=cpu, half=False, inplace=False, train=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=12, verbose=False, workspace=4, nms=False, agnostic_nms=False, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['onnx']
YOLOv5 🚀 2022-2-22 torch 1.11.0 CPU
Fusing layers...
Model Summary: 213 layers, 7225885 parameters, 0 gradients
PyTorch: starting from weights/yolov5s.pt with output shape (1, 25200, 85) (14.8 MB)
ONNX: starting export with onnx 1.14.0...
ONNX: export success, saved as weights/yolov5s.onnx (29.3 MB)
Export complete (2.41s)
Results saved to /home/tuc/yolov5-6.1/weights
Detect: python detect.py --weights weights/yolov5s.onnx
PyTorch Hub: model = torch.hub.load('ultralytics/yolov5', 'custom', 'weights/yolov5s.onnx')
Validate: python val.py --weights weights/yolov5s.onnx
Visualize: https://netron.app
(tc_yolov5) root@rtx2060:/home/tuc/yolov5-6.1#
然后将configs/yolov5.yaml 下的dynamic设为0
执行 ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images 依旧不起作用。
root@rtx2060:/home/tuc/TensorRT_Inference_Demo-main/bin# ./object_detection yolov5 /home/tuc/yolov5-6.1/data/images
[08/16/2023-20:09:42] [I] [TRT] [MemUsageChange] Init CUDA: CPU +547, GPU +0, now: CPU 554, GPU 243 (MiB)
[08/16/2023-20:09:43] [I] [TRT] [MemUsageChange] Init builder kernel library: CPU +273, GPU +68, now: CPU 846, GPU 311 (MiB)
[08/16/2023-20:09:43] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-20:09:43] [I] [TRT] Input filename: ../weights/yolov5/yolov5s.onnx
[08/16/2023-20:09:43] [I] [TRT] ONNX IR version: 0.0.7
[08/16/2023-20:09:43] [I] [TRT] Opset version: 12
[08/16/2023-20:09:43] [I] [TRT] Producer name: pytorch
[08/16/2023-20:09:43] [I] [TRT] Producer version: 1.11.0
[08/16/2023-20:09:43] [I] [TRT] Domain:
[08/16/2023-20:09:43] [I] [TRT] Model version: 0
[08/16/2023-20:09:43] [I] [TRT] Doc string:
[08/16/2023-20:09:43] [I] [TRT] ----------------------------------------------------------------
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:43] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[08/16/2023-20:09:44] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +670, GPU +148, now: CPU 1549, GPU 459 (MiB)
[08/16/2023-20:09:44] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +245, GPU +54, now: CPU 1794, GPU 513 (MiB)
[08/16/2023-20:09:44] [W] [TRT] TensorRT was linked against cuDNN 8.4.1 but loaded cuDNN 8.4.0
[08/16/2023-20:09:44] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[08/16/2023-20:11:29] [I] [TRT] Detected 1 inputs and 7 output network tensors.
[08/16/2023-20:11:29] [I] [TRT] Total Host Persistent Memory: 151744
[08/16/2023-20:11:29] [I] [TRT] Total Device Persistent Memory: 1760768
[08/16/2023-20:11:29] [I] [TRT] Total Scratch Memory: 0
[08/16/2023-20:11:29] [I] [TRT] [MemUsageStats] Peak memory usage of TRT CPU/GPU memory allocators: CPU 0 MiB, GPU 0 MiB
[08/16/2023-20:11:29] [I] [TRT] [BlockAssignment] Algorithm ShiftNTopDown took 16.8801ms to assign 7 blocks to 127 nodes requiring 35635200 bytes.
[08/16/2023-20:11:29] [I] [TRT] Total Activation Memory: 35635200
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in building engine: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/16/2023-20:11:29] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/16/2023-20:11:29] [W] [TRT] The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] Init CUDA: CPU +0, GPU +0, now: CPU 2299, GPU 647 (MiB)
[08/16/2023-20:11:29] [I] [TRT] Loaded engine size: 31 MiB
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
[08/16/2023-20:11:29] [I] [TRT] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +0, now: CPU 0, GPU 0 (MiB)
batch preprocess time: 0.4705ms batch inference time: 0.5246ms batch postprocess time: 3.6264ms
batch preprocess time: 0.3895ms batch inference time: 0.4900ms batch postprocess time: 3.5747ms
Average processing time is 11.3451ms
Average FPS is 88.1440
我暂时也无法定位问题，因为我复现不出来。。
博主，您好，我将我本地的模型和其代码上传到 https://github.com/xunxuntu/check_todo.git 这里了，您有空可以git 下来帮我看看吗 😁😁😁😁😁😁😁😁😁