TensorRT 8.2.2 and 8.2.3 objects are not cleaned properly

[achigin]

Description

TensorRT 8.2.2 and TensorRT 8.2.3 both do not pass sanitizers tests. My assumption is this issue related to smart pointers destruction. Since now you don't need to call destroy function directly and using smart pointers instead. In TensorRT 7.2.3.4 this issue doesn't happen.

Environment

TensorRT Version: 8.2.2.1 and 8.2.3.0 NVIDIA GPU: Tesla V100 NVIDIA Driver Version: 470.57.02 CUDA Version: 11.4.2 and 11.4.3 CUDNN Version: 8.2.1.32 and 8.2.4.14 Operating System: Linux ubuntu 18 DGX Python Version (if applicable): Tensorflow Version (if applicable): PyTorch Version (if applicable): Baremetal or Container (if so, version):

Relevant Files

code to reproduce issue

#include <iostream>
#include <fstream>
#include <memory>
#include <vector>
#include <NvInfer.h>

class Logger : public nvinfer1::ILogger {
  private:
      void log(nvinfer1::ILogger::Severity severity, const char* msg) noexcept override {
          switch (severity) {
              case nvinfer1::ILogger::Severity::kINTERNAL_ERROR:
              case nvinfer1::ILogger::Severity::kERROR:
              case nvinfer1::ILogger::Severity::kWARNING:
                  std::cerr << msg << std::endl;
                  break;
          }
      }
};

int main() {
    Logger logger;
    std::unique_ptr<nvinfer1::IRuntime> infer {nvinfer1::createInferRuntime(logger)};
    std::vector<char> model;
    std::ifstream file_stream("am.trt", std::ios::binary);
    file_stream.seekg(0, file_stream.end);
    const size_t size = file_stream.tellg();
    file_stream.seekg(0, file_stream.beg);
    model.resize(size);
    file_stream.read(model.data(), size);
    std::unique_ptr<nvinfer1::ICudaEngine> engine;
    engine.reset(infer->deserializeCudaEngine(model.data(), model.size()));
    std::cout << "model loaded" << std::endl;
    return 0;
}

Steps To Reproduce

1 building $ g++ main.cpp -I/opt/tensorrt_8.2.3.0_cuda_11.4/include/ -I/usr/local/cuda/include -L/opt/tensorrt_8.2.3.0_cuda_11.4/lib -lnvinfer -g -O0 -fno-omit-frame-pointer -fsanitize=address -fsanitize=undefined 2 running $ ASAN_OPTIONS=protect_shadow_gap=0 ./a.out 3 stdout && stderr

TensorRT was linked against cuBLAS/cuBLAS LT 11.6.5 but loaded cuBLAS/cuBLAS LT 11.6.1
model loaded

=================================================================
==95024==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 72704 byte(s) in 1 object(s) allocated from:
    #0 0x7f9501eeb9c7 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f945f3575ef  (<unknown module>)

Direct leak of 72704 byte(s) in 1 object(s) allocated from:
    #0 0x7f9501eeb9c7 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x7f94711a58cf  (<unknown module>)

SUMMARY: AddressSanitizer: 145408 byte(s) leaked in 2 allocation(s).

If you remove engine.reset... line - no errors.

[mk-nvidia]

Thanks for reporting this issue, we'll act on it.

[201419]

Is there any progress on this issue? Or a temporary fix?

[nvpohanh]

@achigin Could you try TRT 8.4 and see if the issue still exists? Thanks

[achigin]

@nvpohanh Sorry, I'm currently on vacation, will check and report later. Glad for the news at least :)

[syjeon121]

@achigin hi, have you tested it in TRT 8.4? Sorry if you're still on vacation :)

[achigin]

@syjeon121 I'm sorry for such a late response. I was finally on track to test this on TRT 8.4.3.1 + CUDA 11.6 - I can totally confirm, that this issue has been resolved.