TimDettmers
bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
Hi, I'm trying to use the 8-bit optimizer with an A100 on a OpenPBS environment, which means that the machine I install the python virtual environment, with the bitsandbytes lib, doesn't have the GPU, but when I submit the job it does.
Hence, I'm getting this error:
The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
Which makes me think it's because I installed the library on the machine without GPU. How can I install it with GPU support on a machine without GPU? I'm using cu116 btw
PS.When I submit the job, the target machine doesn't have access to internet, so it's not able to fetch the library or anything else online.
Thanks in advance
The binary that is used is determined at runtime. This means in your case there are two modes of failures:
- the CUDA driver is not detected (
libcuda.so) - the runtime library is not detected (
libcudart.so)
Both libraries need to be detected in order to find the right library for the GPU/CUDA version that you are trying to execute against.
Could you please provide use with the output of python -m bitsandbytes for the GPU job that you are trying to run?
I think I'm in a similiar boat trying to get this to engage with the python version of dreambooth-sd
hi, same issue, exact same use-case too haha
===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
For effortless bug reporting copy-paste your error into this form: https://docs.google.com/forms/d/e/1FAIpQLScPB8emS3Thkp66nvqwmjTEgxp8Y9ufuWTzFyr9kJ5AoI47dQ/viewform?usp=sf_link
================================================================================
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/nvidia/lib'), PosixPath('/usr/local/nvidia/lib64')}
"WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:99: UserWarning: /usr/local/nvidia/lib:/usr/local/nvidia/lib64 did not contain libcudart.so as expected! Searching further paths...
f'{candidate_env_vars["LD_LIBRARY_PATH"]} did not contain '
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('module'), PosixPath('//matplotlib_inline.backend_inline')}
"WARNING: The following directories listed in your path were found to "
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('noninteractive SHELL=/bin/bash')}
"WARNING: The following directories listed in your path were found to "
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cuda_setup/paths.py:21: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
"WARNING: The following directories listed in your path were found to "
WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
CUDA SETUP: Loading binary /opt/conda/lib/python3.7/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
/opt/conda/lib/python3.7/site-packages/bitsandbytes/cextension.py:49: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
"The installed version of bitsandbytes was compiled without GPU support. "
I know Im using a cuda-enabled machine
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.85.02 Driver Version: 510.85.02 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA GeForce ... On | 00000000:01:00.0 Off | N/A |
| 30% 31C P8 20W / 350W | 1MiB / 24576MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
I believe this is fixed in the latest version. It prints instructions on how to debug the situation and alternatively prints out compilation instructions which should fix the issue.
I had the same issue, (nvidia-smi had proper output, but was still getting this error)
- I've downloaded cuda drivers from nvidia's website, and followed their instructions:
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-12-0_12.0.1-1_amd64.deb
sudo dpkg -i cuda-12-0_12.0.1-1_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-drivers_525.85.12-1_amd64.deb
wget https://developer.download.nvidia.com/compute/cuda/repos/$distro/$arch/cuda-keyring_1.0-1_all.deb
sudo dpkg -i cuda-keyring_1.0-1_all.deb
sudo apt-key del 7fa2af80
sudo apt-get install cuda
- it failed and messed up my debian package graph, (I've done a bunch of below, it will be different on other systems but just to document)
sudo apt --fix-broken install
sudo systemctl stop nvidia-powerd
sudo apt-get remove nvidia-kernel-common-515
sudo dpkg -r nvidia-compute-utils-525
sudo dpkg -i --force-overwrite /var/cache/apt/archives/nvidia-compute-utils-525_525.85.12-0ubuntu1_amd64.deb
sudo dpkg --configure -a
sudo apt-get install -f
- Once everything is sorted out, rebooted the system:
sudo reboot
Now it works.
Bitsandbytes was not supported windows before, but my method can support windows.（yuhuang） 1 open folder J:\StableDiffusion\sdwebui，Click the address bar of the folder and enter CMD or WIN+R, CMD 。enter，cd /d J:\StableDiffusion\sdwebui 2 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes
3 J:\StableDiffusion\sdwebui\py310\python.exe -m pip uninstall bitsandbytes-windows
4 J:\StableDiffusion\sdwebui\py310\python.exe -m pip install https://github.com/jllllll/bitsandbytes-windows-webui/releases/download/wheels/bitsandbytes-0.41.1-py3-none-win_amd64.whl
Replace your SD venv directory file（python.exe Folder） here（J:\StableDiffusion\sdwebui\py310）
