Does Sapien support cuda rendering

[kenchen3000]

Hi, From Maniskills I see that it support the device type for gpu, cuda and cpu. But in Sapien i can't find cuda related renderer. I only see SapienRenderer and KuafuRenderer. Can you let me know if cuda rendering is supported and where I can find the code for it? thanks!

[fbxiang]

SAPIEN uses Vulkan as the rendering backend, which supports all major GPUs and even CPUs. You may be referring to Vulkan's interop with CUDA, where Vulkan GPU buffers are exposed as CUDA buffers (since Vulkan and CUDA are really just different "front end" for NVIDIA GPU). This is the case for ManiSkill where SAPIEN exposes Vulkan rendered images as CUDA buffers, which are then taken by torch. So it appears to users as rendered by CUDA. KuafuRenderer is now fully integrated into SapienRenderer, and it is removed in SAPIEN 3.

[YouJiacheng]

SAPIEN uses Vulkan as the rendering backend, which supports all major GPUs and even CPUs.

I just tested it. While Vulkan supports CPUs, SAPIEN doesn't. Using Intel CPUs + mesa-vulkan-drivers, vulkaninfo | grep "device" can show

VK_LAYER_MESA_device_select (Linux device selection layer) Vulkan version 1.4.303, layer version 1:
        deviceID          = 0x0000
        deviceType        = PHYSICAL_DEVICE_TYPE_CPU
        deviceName        = llvmpipe (LLVM 20.1.2, 256 bits)

But python -m sapien.example.offscreen will throw an error:

/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/__init__.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  import pkg_resources
/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/_vulkan_tricks.py:73: UserWarning: Failed to find Vulkan ICD file. This is probably due to an incorrect or partial installation of the NVIDIA driver. SAPIEN will attempt to provide an ICD file anyway but it may not work.
  warn(
/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/_vulkan_tricks.py:95: UserWarning: Failed to find glvnd ICD file. This is probably due to an incorrect or partial installation of the NVIDIA driver. SAPIEN will attempt to provide an ICD file anyway but it may not work.
  warn(
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/example/offscreen.py", line 54, in <module>
    main()
  File "/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/example/offscreen.py", line 26, in main
    scene = sapien.Scene()
            ^^^^^^^^^^^^^^
  File "/home/ma-user/test/.venv/lib/python3.12/site-packages/sapien/wrapper/scene.py", line 41, in __init__
    [sapien.physx.PhysxCpuSystem(), sapien.render.RenderSystem()]
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: failed to find a rendering device

Due to cloud provider's restrictions, GPU graphics capabilities are not available, so I can only use CPU rendering as a workaround. But SAPIEN doesn't support it.

[YouJiacheng]

Update: SAPIEN can support CPUs with a small fix. It turns out that _ensure_vulkan_icd causes the problem when nvidia-smi is available while graphics capabilities are not. https://github.com/haosulab/SAPIEN/blob/fd97712362e6f7424952d99507317d2f2606e108/python/py_package/_vulkan_tricks.py#L61-L78

Can be fixed by removing these code or setting VK_ICD_FILENAMES=/usr/share/vulkan/icd.d to make it early return.