Would you please support LoRA training?

[mihara-bot]

Dear authors, Would you please support LoRA training? :) Since I have only 3090 GPUs.

[kawine]

Thanks for the suggestion! This is on our radar and we hope to add support for this over the next few months.

(also please feel free to make a PR if you implement this yourself before then)

[kawine]

Apologies for the very late follow-up to this! The repo now supports LoRA training under the accelerate branch. All you have to do is specify ++model.use_peft=true when you launch the script. You can tweak the lora configs in config/model/base_model.yaml, but by default we use lora_r = 64, lora_alpha = 256, and attach it to all linear modules.

The repo also supports caching of reference model probabilities via ++cache_reference_logprobs=true, so you can reduce memory usage by almost half at the cost of increased run time.

We will merge this into the main branch soon but you can fork the accelerate branch until then.