fairseq2
fairseq2 copied to clipboard
Introduce unsorted_map op
What does this PR do? Please describe: This PR introduces an unsorted_map op; it launches a thread pool that fetches and maps examples in the background.
Does your PR introduce any breaking changes? If yes, please list them: List of all backwards-incompatible changes. N/A
Check list:
- [ ] Was the content of this PR discussed and approved via a GitHub issue? (no need for typos or documentation improvements)
- [ ] Did you read the contributor guideline?
- [ ] Did you make sure that your PR does only one thing instead of bundling different changes together?
- [ ] Did you make sure to update the documentation with your changes? (if necessary)
- [ ] Did you write any new necessary tests?
- [ ] Did you verify new and existing tests pass locally with your changes?
- [ ] Did you update the CHANGELOG? (no need for typos, documentation, or minor internal changes)