xlora
xlora copied to clipboard
EricLBuehler
X-LoRA: Mixture of LoRA Experts
I've trained xlora with mistral 7b base model, it works fine. However, when switching base model to llama2 7b, it encountered an error. This is my code for training. ```...
I saw discussions about training in other issues, and I have run train and inference code successfully. Training code is mainly based on SFTTrainer and I think only next-token prediction...
to train xlora on free collab we need to load a quantized model but currently, xlora does not support the quantized model and layers are not swapping. Please upgrade xlora...
Hello, to obtain this image, what information is needed?
I was trying to use xlora for combining Flan-T5 LoRAs and ran into error within apply_scalings_to_x, does xLoRA support seq2seq models such as Flan-T5 and BART ?
hi, thank you for your excellent work. Do you have any plans to share the training code? i want to reproduce the training but raises error as followers `RuntimeError: Trying...
There's an error while I ran the generation code. For example, xlora_model.generate(torch.randint(100, 1000, (1, 8)).to('cuda'), max_new_tokens=1) throws: RuntimeError: The expanded size of the tensor (16) must match the existing size...
I am using a Quantised Qwen VL 2B model, I loaded that as a base model then passed the same to initialise a trained xlora model as follows: ``` base_model...
