Alex

@UranusSeven , am also running 2*3090, how did you force-distribute memory across both cards for fine-tuning? tried setting max-memory and updated the new finetune.py from git but no luck... still...

I've been working on exactly the same thing across a few domains though it's not all 'open' but could probably be adapted with enough work. Been finding a lot of...

Might even reach out to the open-assistant people about possible collaboration

+1 I'd love to see this too. If helpful, I have access to a sufficiently capable machine (Ubuntu/28c/168Gb/132GbSwap/NVlink2x3090/48gb-Vram) and would be willing to provide the compute if anyone can draft...

Y'all: This shouldn't be difficult. I finetuned the 30B 8-bit Llama with Alpaca Lora in about 26 hours on a couple of 3090's with good results. The 65B model quantized...