bitsandbytes-win-prebuilt icon indicating copy to clipboard operation
bitsandbytes-win-prebuilt copied to clipboard

Update for newest bitsandbytes version

Open aicrumb opened this issue 2 years ago • 0 comments

My methodology was pip installing bitsandbytes and then applying the fixes in this repo. Problem is the second line to fix doesn't exist anymore and was moved to ~line 122 of the first file (main.py in cuda_setup).

After making the suggested fix to that line (and the line two after it, since it's the same) instead, I have it working on the latest (stable) pytorch version as of writing and cudatoolkit 11.6.2 with huggingface's load_in_8bit param on GPT2, tested just inference.

aicrumb avatar Mar 11 '23 22:03 aicrumb