Nsight version deprecation? NVTX3 errors.
Describe the bug Nsight breaking changes.
The implementation of NVTX from Cuda 11.x to Cuda 12.x seem to have been changed. Compilation on Cuda 12.x (seem like >12.2) failed with NVTX errors.
To Reproduce Steps to reproduce the behavior. If the code is not attached and cannot be reproduced easily, the bug report will be closed without any comments.
Run py setup.py install
Expected behavior A clear and concise description of what you expected to happen.
Successful installation after suggested changes in #607
Desktop (please complete the following information):
- OS: Ubuntu 22.04 on WSL2
- Python version: 3.10.11
- Pytorch version: 2.5.1+cu124
- CUDA version: 12.6
- NVIDIA Driver version: 561.17
- Minkowski Engine version 02fc608
- Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of
python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)
wget -q https://raw.githubusercontent.com/NVIDIA/MinkowskiEngine/master/MinkowskiEngine/diagnostics.py ; python diagnostics.py
==========System========== Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35 DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.5 LTS" 3.10.12 (main, Nov 6 2024, 20:22:13) [GCC 11.4.0] ==========Pytorch========== 2.5.1+cu124 torch.cuda.is_available(): True ==========NVIDIA-SMI========== /usr/lib/wsl/lib/nvidia-smi Driver Version 561.17 CUDA Version 12.6 VBIOS Version 90.17.31.00.23 Image Version G001.0000.02.04 GSP Firmware Version N/A ==========NVCC========== /usr/local/cuda-12.6/bin/nvcc nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2024 NVIDIA Corporation Built on Tue_Oct_29_23:50:19_PDT_2024 Cuda compilation tools, release 12.6, V12.6.85 Build cuda_12.6.r12.6/compiler.35059454_0 ==========CC========== /usr/bin/c++ c++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 Copyright (C) 2021 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
==========MinkowskiEngine========== MinkowskiEngine not installed
Additional context Add any other context about the problem here.
Seem this repository is not updated.
For now you need to manually reimplement the NVTX events/markings on this library,
or just disable it with -DNVTX_DISABLE
sorry how did you handle the NVTX errors? I followed https://github.com/NVIDIA/MinkowskiEngine/issues/601 but I still have the nvtx3 errors.
@RivettiLuciano, sorry, I decided not to pursue this path and dropped the project.
Haven't looked into the code yet, but assuming Visual Studio--and by extension, IntelliSense-- can point out the errors, you may be able to reimplement the NVTX events/markings.
NVTX is telemetry for profiling and such, if you don't need it, use the -DNVTX_DISABLE variable.
Hi @ha-ves I tried adding to the setup.py
extra_compile_args = { "cxx": [ARGS, "-DNVTX_DISABLE"] }
but it doesnt work i still have the errors with nvtx3
@RivettiLuciano I believe the flag/variable is for nvcc, try put it in NVCC_FLAGS
@ha-ves that was correct. I could install Monkoski thank you very much for your answer
Thank you all for the pointers!
I was finally able to work around the NVTX3 header collisions by completely disabling the cuDF NVTX code paths at compile time. In my setup.py, I added the NVTX_DISABLE macro both as a define and in the compiler flags:
ext_modules = [
Extension(
name="MinkowskiEngineBackend._C",
sources=[…],
+ define_macros=[('NVTX_DISABLE', None)],
extra_compile_args={
'cxx': CC_FLAGS + ['-DNVTX_DISABLE'],
'nvcc': NVCC_FLAGS + ['-DNVTX_DISABLE'],
},
libraries=libraries,
),
]
After cleaning and reinstalling:
python setup.py clean --all
python setup.py install
the build succeeds without any more ambiguous domain/category errors.
Hope this helps anyone else hitting the same CUDA 12.x + NVTX3 clash!
Thank you all for the pointers!
I was finally able to work around the NVTX3 header collisions by completely disabling the cuDF NVTX code paths at compile time. In my
setup.py, I added theNVTX_DISABLEmacro both as a define and in the compiler flags:ext_modules = [ Extension( name="MinkowskiEngineBackend._C", sources=[…],
), ] After cleaning and reinstalling:define_macros=[('NVTX_DISABLE', None)], extra_compile_args={ 'cxx': CC_FLAGS + ['-DNVTX_DISABLE'], 'nvcc': NVCC_FLAGS + ['-DNVTX_DISABLE'], }, libraries=libraries,python setup.py clean --all python setup.py install the build succeeds without any more ambiguous
domain/categoryerrors.Hope this helps anyone else hitting the same CUDA 12.x + NVTX3 clash!
Thank u mate! Really helpful!