bitsandbytes icon indicating copy to clipboard operation
bitsandbytes copied to clipboard

NameError: name 'str2optimizer8bit_blockwise' is not defined

Open utopictown opened this issue 3 years ago • 0 comments

i got this error when running dreambooth training from here

chiron    | /usr/local/lib/python3.9/site-packages/bitsandbytes/cuda_setup/paths.py:27: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('https'), PosixPath('//github.com/pypa/get-pip/raw/5eaac1050023df1f5c98b173b248c260023f2278/public/get-pip.py')}
chiron    |   warn(
chiron    | CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
chiron    | /usr/local/lib/python3.9/site-packages/bitsandbytes/cuda_setup/paths.py:27: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
chiron    |   warn(
chiron    | WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
chiron    | CUDA SETUP: Loading binary /usr/local/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
chiron    | /usr/local/lib/python3.9/site-packages/bitsandbytes/cextension.py:48: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
chiron    |   warn(
Steps:   0%|          | 0/800 [00:01<?, ?it/s, loss=0.183, lr=5e-6]Traceback (most recent call last):
chiron    |   File "/app/train_dreambooth.py", line 592, in <module>
chiron    |     main()
chiron    |   File "/app/train_dreambooth.py", line 560, in main
chiron    |     optimizer.step()
chiron    |   File "/usr/local/lib/python3.9/site-packages/accelerate/optimizer.py", line 140, in step
chiron    |     self.optimizer.step(closure)
chiron    |   File "/usr/local/lib/python3.9/site-packages/torch/optim/lr_scheduler.py", line 65, in wrapper
chiron    |     return wrapped(*args, **kwargs)
chiron    |   File "/usr/local/lib/python3.9/site-packages/torch/optim/optimizer.py", line 113, in wrapper
chiron    |     return func(*args, **kwargs)
chiron    |   File "/usr/local/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
chiron    |     return func(*args, **kwargs)
chiron    |   File "/usr/local/lib/python3.9/site-packages/bitsandbytes/optim/optimizer.py", line 265, in step
chiron    |     self.update_step(group, p, gindex, pindex)
chiron    |   File "/usr/local/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
chiron    |     return func(*args, **kwargs)
chiron    |   File "/usr/local/lib/python3.9/site-packages/bitsandbytes/optim/optimizer.py", line 506, in update_step
chiron    |     F.optimizer_update_8bit_blockwise(
chiron    |   File "/usr/local/lib/python3.9/site-packages/bitsandbytes/functional.py", line 858, in optimizer_update_8bit_blockwise
chiron    |     str2optimizer8bit_blockwise[optimizer_name][0](
chiron    | NameError: name 'str2optimizer8bit_blockwise' is not defined
Steps:   0%|          | 0/800 [00:01<?, ?it/s, loss=0.183, lr=5e-6]
chiron    | Traceback (most recent call last):
chiron    |   File "/usr/local/bin/accelerate", line 8, in <module>
chiron    |     sys.exit(main())
chiron    |   File "/usr/local/lib/python3.9/site-packages/accelerate/commands/accelerate_cli.py", line 43, in main
chiron    |     args.func(args)
chiron    |   File "/usr/local/lib/python3.9/site-packages/accelerate/commands/launch.py", line 837, in launch_command
chiron    |     simple_launcher(args)
chiron    |   File "/usr/local/lib/python3.9/site-packages/accelerate/commands/launch.py", line 354, in simple_launcher
chiron    |     raise subprocess.CalledProcessError(returncode=process.returncode, cmd=cmd)

result from python -m bitsandbytes

++++++++++++++++++++++++++ OTHER +++++++++++++++++++++++++++
COMPILED_WITH_CUDA = False
COMPUTE_CAPABILITIES_PER_GPU = ['7.0']
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++ DEBUG INFO END ++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Running a quick check that:
    + library is importable
    + CUDA function is callable

name 'str2optimizer32bit' is not defined

run inside docker on VM linux ubuntu 20, 64GB, Intel(R) Xeon(R) CPU @ 2.20GHz, Nvidia GV100GL [Tesla V100 SXM2 16GB]

utopictown avatar Oct 14 '22 20:10 utopictown