open3d.cpu not found after successful py wheel build

[Coelhomatias]

Checklist

• [X] I have searched for similar issues.
• [x] For Python issues, I have tested with the latest development wheel.
• [X] I have checked the release documentation and the latest documentation (for main branch).

Steps to reproduce the issue

I first cloned Open3D by:

git clone https://github.com/isl-org/Open3D.git
cd Open3D

Followed the docker to install dependencies for python and jupyter

Then, I build Open3D (on Ubuntu 22.04, with CUDA 11.8 and pytorch=2.1.2) with:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_LIBREALSENSE=ON -DBUILD_CUDA_MODULE=ON -DBUILD_COMMON_ISPC_ISAS=ON -DCMAKE_CUDA_ARCHITECTURES="90;89;86;80" -DGLIBCXX_USE_CXX11_ABI=OFF -DBUILD_PYTORCH_OPS=ON -DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=/home/coelhomatias/Open3D-ML -DBUILD_UNIT_TESTS=ON -DBUILD_BENCHMARKS=ON -DBUILD_EXAMPLES=ON -DCMAKE_INSTALL_PREFIX=$HOME/open3d_install -DBUILD_WEBRTC=ON -DBUILD_JUPYTER_EXTENSION=ON ..
make -j$(nproc)
make pip-package -j$(nproc)

In another computer with a fresh install of Ubuntu22.04

• Create conda environment with miniconda
• Install correct pytorch version (2.1.2 for cu118)
• run python -c "import open3d"

Error message

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
Cell In[2], line 1
----> 1 import open3d

File ~/fruitvista_open3d/.conda/lib/python3.11/site-packages/open3d/__init__.py:93
     87         warnings.warn(
     88             "Open3D was built with CUDA support, but Open3D CUDA Python "
     89             "binding library not found! Falling back to the CPU Python "
     90             "binding library.", ImportWarning)
     92 if __DEVICE_API__ == "cpu":
---> 93     from open3d.cpu.pybind import (core, camera, data, geometry, io, pipelines,
     94                                    utility, t)
     95     from open3d.cpu import pybind
     98 def _insert_pybind_names(skip_names=()):

ModuleNotFoundError: No module named 'open3d.cpu'

Open3D, Python and System information

- Operating system: Ubuntu 22.04 / macOS 10.15 / Windows 10 64-bit
- Python version: Python 3.11.7 (main, Dec 15 2023, 18:12:31) [GCC 11.2.0]
- Open3D version: 0.18
- System architecture: x86
- Is this a remote workstation?: WSL
- How did you install Open3D?: pip (after build wheel from source)
- Compiler version (if built from source): gcc 11.8 / clang 11.8

Additional information

The presented error was actually the second encountered. The first error was related to libunwind.so.1. This file was non existing even after sudo apt-get install libunwind-dev, however, a libunwind.so.8 existed and the error was "fixed" by doing a symbolic link between the files.

[saurabheights]

You have built using cmake .. DBUILD_CUDA_MODULE=ON ... which means open3d should be built with cuda. The issue is in Open3d init.py file here, where _pybind_cuda.open3d_core_cuda_device_count() fails and following code to set __DEVICE_API__ = "cuda" is never run. You can also pass -Wdefault to python to see the warnings.

I usually reload NVIDIA drivers or reboot to circumvent this issue.

sudo rmmod nvidia_uvm
sudo modprobe nvidia_uvm

P.S. I did a test with the cuda docker image and instead of running Open3D code, I compiled deviceQuery. It also reported some error, so the issue wasn't in that case in Open3D.

@ssheorey Wouldn't it be better to terminate at https://github.com/isl-org/Open3D/blob/main/python/open3d/init.py#L78 since the code will crash anyways. We can covnert warning into an error message with more clear instructions.

[Coelhomatias]

I did some additional tests today and found out some important things. The issue I was having with libunwind was related to being inside WSL. Some dependencies don't come pre installed and I was able to fix this issue with sudo apt-get install ffmpeg libsm6 libxext6 -y. From my very little understanding this has something to do with cv2.

The error message I provided above was the output of running inside a jupyter cell. I get no error when running inside a python script or with the python -c command. So I think this is related to jupyter. However, I made sure to enable jupyter in the build process and installed all dependencies. Why is this happening?

Below is a picture showing this happening

[sitic]

What's the current working directory (os.getcwd()) of that jupyter notebook? If it works in a terminal but not in that notebook there might be a subfolder open3d with an __init__.py but none of the .so libraries there which python tries to import.

At least that's a common issue that I've run into, I'm however a bit confused because the screen shot shows the exception being raised in the installed site-packages folder. Running o3d.__file__ after the successful import in Python will show where the correct version is located, compare that against the path the jupyter kernel tries to import.

[Neflux]

I successfully built Open3D 0.18.0 from source on the CUDA-gl dockerhub image on a conda environment activated during build time, and I get the same error if I try to test it at the end of the build. Nonetheless, it works just fine if I just do it manually from the container. I documented it in this other issue. Maybe my finding can help you advance on this issue in some way?

[JChrysanthemum]

Great! @saurabheights did fix this annoying problem.

For open3d+cuda+torch, I tried many python versions, docker, or install from source, but all failed with open3d.cpu not found. Everything works after a reboot.