ColossalAI icon indicating copy to clipboard operation
ColossalAI copied to clipboard
verified publisher icon hpcaitech

[BUG]: LoRA does not support the training of reward models

Open young-chao opened this issue 2 years ago β€’ 1 comments

πŸ› Describe the bug

image

Environment

env

OS：ubuntu 20.04 GPU：4 x A10 python==3.9.0 torch==1.13.1-cu116 colossalai==0.2.5

command

python train_reward_model.py --pretrain "bigscience/bloom-560m" --lora_rank 16

young-chao avatar Feb 22 '23 09:02 young-chao

I am very confused why the training encounters errors after setting lora_rank, whether it is the wrong way of my use。

young-chao avatar Feb 22 '23 09:02 young-chao

@ht-zhou I read your modifications, I don't think these modifications can fix the problem, and I noticed that the latest main version of ColossalAI removed the use of LoRA during rm model training, and now I think ColossalAI does not support LoRA at all, right?

young-chao avatar Mar 05 '23 14:03 young-chao

image

young-chao avatar Mar 05 '23 14:03 young-chao

Bot detected the issue body's language is not English, translate it automatically. πŸ‘―πŸ‘­πŸ»πŸ§‘β€πŸ€β€πŸ§‘πŸ‘«πŸ§‘πŸΏβ€πŸ€β€πŸ§‘πŸ»πŸ‘©πŸΎβ€πŸ€β€πŸ‘¨πŸΏπŸ‘¬πŸΏ

image

Issues-translate-bot avatar Mar 05 '23 14:03 Issues-translate-bot