PKU-Aligner

是的，对齐器也是在全参数微调，通过Q-A-C来进行残差微调。 训练一个7B的模型，和SFT所需要的资源是一致的，你可以考虑lora或者更小尺寸的aligner，比如可以拿qwen1.5-2B来训练对齐器，是可以直接在3090上训练了。 Yes, aligners are also fine-tuned with full-parameter fine-tuning, using Q-A-C for residual fine-tuning. Training a 7B model requires the same resources as SFT, you can consider using lora...