node.output().size() == 1 && "TensorRT does not support the indices output in MaxPool!" failure of TensorRT 8.6 when running ONNX model on GPU RTX 3090

[lhai37]

Description

I tried to run the attached ONNX model through trtexec, but it fails with the error: node.output().size() == 1 && "TensorRT does not support the indices output in MaxPool!". This is because MaxPool is being used in the model to produce 2 outputs: the values and indices. Upon checking the officially supported ONNX operators I see that MaxPool layer with indices output is not supported even in TensorRT 10 EA. Is there any plan to support this or workaround for this issue?

Environment

TensorRT Version: 8.6

NVIDIA GPU: RTX 3090

NVIDIA Driver Version: 545

CUDA Version: 11.6

CUDNN Version: 8.8

Operating System: Ubuntu 20.04 64-bit

Python Version (if applicable):

Tensorflow Version (if applicable):

PyTorch Version (if applicable):

Baremetal or Container (if so, version):

Relevant Files

Model link: https://drive.google.com/file/d/1ZvhQiG9NT92CbVK7oAznBR_rAlF5AzzF/view?usp=sharing

Steps To Reproduce

Run trtexec --onnx=model.onnx

Commands or scripts: Run trtexec --onnx=model.onnx

Have you tried the latest release?: N/A

Can this model run on other frameworks? N/A

[lix19937]

ref
https://docs.nvidia.com/deeplearning/tensorrt/operators/docs/Pooling.html https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Graph/Layers.html#ipoolinglayer

Computes a per-channel pooling using a sampling window on the input tensor into an output tensor. The supported sampling window shapes are 2-D or 3-D.
If you need 2 outputs of pooling layer, i think for now you’d need to use a plugin.

[lhai37]

Is there any existing plugin for MaxPool with 2 outputs? I looked through the default plugins that are shipped with TensorRT and didn't see any there, but perhaps someone has already created this elsewhere.

[lix19937]

You can write a custom plugin by yourself.

[bs1119]

Is there any existing plugin for MaxPool with 2 outputs? I looked through the default plugins that are shipped with TensorRT and didn't see any there, but perhaps someone has already created this elsewhere.

Were you able to solve the issue? If so, can you please share the plugin? Thanks.

[lix19937]

You can upload your torch network code fragment about maxpool.

[kevin-chau]

I'm having this issue with a pytorch ENET model I exported to ONNX. Has somebody written a custom plugin for the MaxPool layer with indices output? Would love to be able to move past this issue!

[ducknificient]

ref https://docs.nvidia.com/deeplearning/tensorrt/operators/docs/Pooling.html https://docs.nvidia.com/deeplearning/tensorrt/api/python_api/infer/Graph/Layers.html#ipoolinglayer

Computes a per-channel pooling using a sampling window on the input tensor into an output tensor. The supported sampling window shapes are 2-D or 3-D. If you need 2 outputs of pooling layer, i think for now you’d need to use a plugin.

the link provided returning Page Not Found, thanks

[lix19937]

I'm having this issue with a pytorch ENET model I exported to ONNX. Has somebody written a custom plugin for the MaxPool layer with indices output? Would love to be able to move past this issue!

Usually maxpool layer will fusion with downstream operator, Of course, writing an implementation using CUDA alone is also very simple.

[kevin-chau]

@lix19937 Could you provide some sample code for how to implement MaxPool with CUDA?