NeMo icon indicating copy to clipboard operation
NeMo copied to clipboard
verified publisher icon NVIDIA

Fix FSDP SHARP integration

Open janEbert opened this issue 2 years ago • 4 comments

The attribute was never implemented.

What does this PR do ?

For NeMo NLP, the FSDP sharding strategy cannot be used currently because the sharp attribute for the NLPFSDPStrategy was not actually implemented in https://github.com/NVIDIA/NeMo/pull/7793, even though it is queried.

Collection:

  • NLP

Changelog

  • Implement missing NLPFSDPStrategy.sharp attribute.

Jenkins CI

To run Jenkins, a NeMo User with write access must comment jenkins on the PR.

Before your PR is "Ready for review"

Pre checks:

  • [X] Make sure you read and followed Contributor guidelines
  • [ ] Did you write any new necessary tests?
  • [ ] Did you add or update any necessary documentation?
  • [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • [ ] Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • [ ] New Feature
  • [X] Bugfix
  • [ ] Documentation

Who can review?

Anyone in the community is free to review the PR once the checks have passed. Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to #7793 (pull request)

janEbert avatar Jan 13 '24 21:01 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Jan 28 '24 01:01 github-actions[bot]

Bla.

janEbert avatar Jan 28 '24 20:01 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Feb 13 '24 01:02 github-actions[bot]

Bla.

janEbert avatar Feb 13 '24 08:02 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Feb 29 '24 01:02 github-actions[bot]

Bla.

janEbert avatar Feb 29 '24 07:02 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Mar 16 '24 01:03 github-actions[bot]

Bla.

janEbert avatar Mar 18 '24 08:03 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Apr 02 '24 01:04 github-actions[bot]

Bla.

janEbert avatar Apr 02 '24 07:04 janEbert

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions[bot] avatar Apr 17 '24 01:04 github-actions[bot]

Bla.

janEbert avatar Apr 22 '24 07:04 janEbert

Independently fixed in dd69c7a7d.

janEbert avatar May 29 '24 12:05 janEbert