Jianbin Chang

Questions about log interpretation, seems paradoxical

6

This line of log confuses me, my batch size is 513 and iteration time is 98.83, so the throughput should be 5.19. Obviously, the logs of iteration time and throughput...

Coord check looks good, but μTransfer is not working as expected

4

Hello, μP team! Very excited to see you open source your excellent work! I was looking to apply μP on our work, and on Megatron-DeepSpeed I modified the training script...

buildkite ci process stuck on rust related install, try to fix this. BTW: CI Agent may have some problems now.

Whether Distributed Optimizer wrapper meets the requirements?

1

Hi, there. I want to ask whether the current rules allow the use of distributed optimizer wrappers like hvd.DistributedOptimizer ? I see that we have a [list of optimizers allowed...

> [!IMPORTANT] > The `Update branch` button must only be pressed in very rare occassions. > An outdated branch is never blocking the merge of a PR. > Please reach...

Draft: disable m-fsdp hooks for torch compile compatible

1

# What does this PR do ? :warning: For major changes (either in lines of code or in its impact), please make sure to first share discuss a design-doc with...

Draft: Megatron-FSDP and PP compatible support

4

# What does this PR do ? :warning: For major changes (either in lines of code or in its impact), please make sure to first share discuss a design-doc with...