BarfingLemurs

Results 59 comments of BarfingLemurs

CUDA out of memory. Tried to allocate 34.00 MiB

>do i need to edit the start web ui for it to work? adding settings in the start_linux.sh script seems to prevent me from starting the program alltogether. It might...

Improve memory management

set it to 4. It doesn't come fine-tuned I think. You have to do it all yourself. I had issue running pylymion6b on 4gb until I did 2, and lowered...

4-bit: "Allocator: not enough memory: you tried to allocate 35389440 bytes."

The final line indicates you are out of DRAM (regular ram)to load the model, have you tried using a pagefile?

4-bit: "Allocator: not enough memory: you tried to allocate 35389440 bytes."

well, It only loads your model into ram initially, then loads it into vram.

Llama 4-bit install instructions no longer work (CUDA_HOME environment variable is not set)

I just installed using this method, setup.py didn't work for me https://github.com/oobabooga/text-generation-webui/issues/177#issuecomment-1464844721 its pre-assembled

Llama 4-bit install instructions no longer work (CUDA_HOME environment variable is not set)

@oobabooga could you distribute the .whl file so we do not have do follow the whole process? This is for WSL on windows, which would be the official recommended method...

Llama 4-bit install instructions no longer work (CUDA_HOME environment variable is not set)

Thanks, but I am hoping to use other people's .whls as I do take a while to gather and follow the build process.

Llama 4-bit install instructions no longer work (CUDA_HOME environment variable is not set)

@jllllll do you have a .whl file? I'm stuck on certain issues which I'm unsure about. I followed through on aregular installation process on WSL, hoping the gpu could be...

Llama 4-bit install instructions no longer work (CUDA_HOME environment variable is not set)

@jllllll I really appreciate that, thanks.

[FIX] fail to load LoRA weights, fail to load LoRA weights in 4-bit, fail to generate text with LoRA in 8-bit, UnboundLocalError: local variable 'new_module' referenced before assignment, ValueError: We need an `offload_dir`, AttributeError: 'NoneType' object has no attribute 'device'

have any of you tested https://huggingface.co/chansung/alpaca-lora-30b?