bitsandbytes-win-prebuilt
bitsandbytes-win-prebuilt copied to clipboard
Update for newest bitsandbytes version
My methodology was pip installing bitsandbytes and then applying the fixes in this repo. Problem is the second line to fix doesn't exist anymore and was moved to ~line 122 of the first file (main.py in cuda_setup).
After making the suggested fix to that line (and the line two after it, since it's the same) instead, I have it working on the latest (stable) pytorch version as of writing and cudatoolkit 11.6.2 with huggingface's load_in_8bit param on GPT2, tested just inference.