dmahan93
dmahan93
should #1214 get closed and pulled into this?
https://docs.gitlab.com/ee/api/ it has an API available, and includes diffs: https://docs.gitlab.com/ee/api/commits.html#get-the-diff-of-a-commit
Also builds off https://github.com/EleutherAI/gpt-neox/pull/1240 and https://github.com/EleutherAI/gpt-neox/pull/1239 since the packing implementations/chat templating items are much more useful for DPO.
FYI Does not support pipeline parallelism
@Quentin-Anthony probably merge conflicts and other stuff, it's v untested, but it sounded like it would be better to have something than wait around so it's a bit rushed so...
@Quentin-Anthony should be ready for a review 🙌