[wip] enable FastPersist to (potentially) speed up disk saving in group offloading

Open sayakpaul opened this issue 8 months ago • 1 comments

What does this PR do?

FastPersist and DeepNVMe: https://github.com/deepspeedai/DeepSpeedExamples/tree/master/deepnvme

Blog post: https://pytorch.org/blog/deepnvme-affordable-i-o-scaling-for-deep-learning-applications/

⛔️ Currently not ready for reviews.

Jun 19 '25 13:06 sayakpaul

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Jun 19 '25 13:06 HuggingFaceDocBuilderDev

Sadly, this won't go much further at this point. So, closing it. But keeping the branch open.

Jul 17 '25 20:07 sayakpaul