pyopencl icon indicating copy to clipboard operation
pyopencl copied to clipboard

pyopencl does not respect OCL_ICD_FILENAMES

Open bjourne opened this issue 1 year ago • 3 comments

MWE:

$ python -c "import pyopencl; print(pyopencl.get_platforms())"                                                                                                                                                               
[<pyopencl.Platform 'NVIDIA CUDA' at 0x5b7c78be31c0>]
$ clinfo -l                                                                                                                                                                                                                  
Platform #0: Intel(R) FPGA Emulation Platform for OpenCL(TM)
 `-- Device #0: Intel(R) FPGA Emulation Device
Platform #1: Intel(R) OpenCL
 `-- Device #0: Intel(R) Xeon(R) W-2245 CPU @ 3.90GHz
Platform #2: NVIDIA CUDA
 `-- Device #0: Quadro P400

So pyopencl does not find platforms whose shared objects are found in the OCL_ICD_FILENAMES environment variable. I'm not sure why since get_platforms simply wraps clGetPlatformIDs, doesn't it?

bjourne avatar Oct 29 '24 00:10 bjourne

Apparently this is because of the ICD loader pyopencl uses. Khronos own ICD loader supports environment variables but ocl-icd does not: https://github.com/KhronosGroup/OpenCL-ICD-Loader Looking at repo activity it seems ocl-icd doesn't get many updates.

bjourne avatar Oct 29 '24 00:10 bjourne

I'm confused here.

  • PyOpenCL is not specific to any ICD loader, it works with Khronos and other loaders. The wheel also does not ship a specific loader.
  • For user convenience, the wheels do include an ICD loader: that's ocl-icd on Windows and Mac, and Khronos on Windows. You can always build from source and avoid this.

If you need a Linux/Mac wheel with Khronos, I'm happy to consider PRs that either replace ocl-icd or offer Khronos as an alternative.

inducer avatar Oct 30 '24 17:10 inducer

Ah, ok. I think I understand what is going on. The pip package puts a libOpenCL.so file in a ".libs" directory which shadows the system-installed libOpenCL.so. This libOpenCL.so comes from ocl-icd and not from Khronos official ICD loader. Shadowing system libraries is always dubious. Honestly, on Linux not including any ICD loader would be better since the user likely already has one installed. And if they don't it's just one "apt-get install ocl-icd" away. If not that then swapping ocl-icd with Kronos' ICD loader would be good. It appears to be more feature-complete and better maintained.

bjourne avatar Oct 31 '24 12:10 bjourne