ray wan

icon company The University of Hong Kong icon location Hong Kong icon location Stay Hungry, Stay Foolish

Results 1 issues of ray wan

[QUESTION] Why enable `non_blocking=True` when doing synchronous D2H?

2 comment

The comment on line 76 of `filesystem_async.py` indicates that Megatron performs synchronous Device-to-Host (D2H) transfers for checkpointing. However, on line 94, the code enables non_blocking=True during these transfers ([code link](https://github.com/NVIDIA/Megatron-LM/blob/main/megatron/core/dist_checkpointing/strategies/filesystem_async.py#L94))....

stale