Jianing Qi
Jianing Qi
> I encountered this error while resuming the checkpoint of Lora training. I found that this is basically due to the old version of Transformers that LLaVA is using. Please...
I tested with students and teachers so received some feedbacks. It would be great for collaborative work for the in room iframe webpage, so everyone can make adjustments on it....
> @sigjhl their fork is public here: https://github.com/user074/unsloth yeah basically you can pass an argument of `sequence_classification = True, num_labels = YOUR_NUMBER_OF_LABELS` when you initialize the unsloth. an example is...
Not for task but for LoRA, I followed #729 to save it. Able to load the LoRA weights after following #1200
Okay. I think I solved it through change the dtype in cli.py, basically I changed `dtype=torch.float16` to `dtype=torch.bfloat16`. I think you also can change it to float32. Also you need...
I guess it kinda work for the full_finetune_single_device.py. I will update after I finish the training to see how it goes
I can have a check on the distributed case but probably after mid November
Thanks @ebsmothers. I will update it by the weekend. Additional note: I can try to do the same implementation for the distributed version but i do not have enough GPUs...
> Hi @user074 just wanted to check in if you're still working on this. If not let me know and one of us can help to take it over the...
> Thanks, I think this is much cleaner! I left one more comment, can you run a test with no LR scheduler (running manually is fine) just to make sure...