Gal Hubara-Agam
Gal Hubara-Agam
Reproduced on latest TRT version. ONNX conversion is issuing a warning "Warning: ONNX Preprocess - Removing mutation from node aten::add_ on block input: 'bn1.num_batches_tracked'. This changes graph semantics." . Should...
Linking similar issue #1771
> set ALLOW_RELEASED_ONNX_OPSET_ONLY=0 and run again Tried that locally: ```export ALLOW_RELEASED_ONNX_OPSET_ONLY=0 pytest onnx/test/test_backend_onnxruntime.py::OnnxBackendNodeModelTest::test_cast_DOUBLE_to_FLOAT16_cpu``` Result is the same: ```__________________________________________________________________________________________________________________ OnnxBackendNodeModelTest.test_cast_DOUBLE_to_FLOAT16_cpu __________________________________________________________________________________________________________________ onnx/test/test_backend_onnxruntime.py:71: in _create_inference_session session = ort.InferenceSession(model.SerializeToString(), providers=providers) ../../.local/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:419: in __init__...
Hi @xadupre, yes, this is in progress right now. I'm not at liberty to discuss the internal plans publicly, but you can review our current unit testing for ONNX export...
TransformerEngine ONNX export is based on TorchScript export. The latest ONNX opset supported in TorchScript is opset 18. Since FP8 data types were first introduced in opset 19, exporting FP8...
Thanks @yuanyao-nv . This issue should be filed under ModelOpt - https://github.com/NVIDIA/TensorRT-Model-Optimizer/issues
@WhiteTeaDragon sorry, I missed that. We'll discuss this internally and get back to you. ModelOpt is the recommended approach. However, any other method that produces a valid ONNX file with...
I think it's more beneficial to keep this discussion in one place - https://github.com/NVIDIA/TensorRT-Model-Optimizer/issues/80. Using ModelOpt for quantization is the recommended approach. Any updates/fixes for the quantization method should go...
Hey @YixuanSeanZhou, I was unable to reproduce the accuracy issue you've observed. Tried both TRT 8.6.1 (NGC image tensorrt-23.09-py3) and TRT 10.6 (NGC image tensorrt-24.10-py3), I don't see any accuracy...
@YixuanSeanZhou if there's no update until 12/12 we'll close this ticket.