openchat icon indicating copy to clipboard operation
openchat copied to clipboard
verified publisher icon imoneoi

Does openchat support LORA finetune?

Open linggong2023 opened this issue 2 years ago • 1 comments

linggong2023 avatar Jul 08 '23 23:07 linggong2023

Currently, only full weight fine-tuning is supported.

imoneoi avatar Jul 09 '23 05:07 imoneoi

Closing this issue now. If you have further questions, plz re-open.

imoneoi avatar Jul 21 '23 09:07 imoneoi

lora or qlora is necessary for training a 70b model on a single machine. I'm very curious about a 70b openchat model.

skepsun avatar Aug 30 '23 07:08 skepsun