yechenzhi

DPO supports multi-device training

11

As previously discussed, we will support the parallel training of DPO. However, it seems that the content of the parallel training config file is almost the same as that of...

[Question] Does sequence parallelism effectively reduce GPU memory in forward pass?

1

When setting max_response_length >= 16k, I'm encountering OOM errors even with ulysses_sequence_parallel_size >= 2 when I set use_dynamic_bsz = False. ``` for epoch in range(self.config.ppo_epochs): for batch_idx, data in enumerate(dataloader):...