streaming
streaming copied to clipboard
tag shared and temp files with username
Description of changes:
- Change StreamingDataset to make a temporary directory in
tmp/<username>/streamingrather thantmp/streaming. - Change the shared memory prefix to
<username>_<old_prefix>
These changes partition shared resources on a per-user basis to prevent permissions errors when multiple users are working on the same system.
Issue #429
- Fixes #429
Merge Checklist:
Put an x without space in the boxes that apply. If you are unsure about any checklist, please don't hesitate to ask. We are here to help! This is simply a reminder of what we are going to look for before merging your pull request.
General
- [x] I have read the contributor guidelines
- [ ] This is a documentation change or typo fix. If so, skip the rest of this checklist.
- [ ] I certify that the changes I am introducing will be backward compatible, and I have discussed concerns about this, if any, with the MosaicML team.
- [ ] I have updated any necessary documentation, including README and API docs (if appropriate).
Tests
- [x] I ran
pre-commiton my change. (check out thepre-commitsection of prerequisites) - [ ] I have added tests that prove my fix is effective or that my feature works (if appropriate).
- [x] I ran the tests locally to make sure it pass. (check out testing)
- [ ] I have added unit and/or integration tests as appropriate to ensure backward compatibility of the changes.
How can we do this while still keeping the local dir collision check across all users?