LLNL
Changes to support Node Local extents.
Description
Currently, unifyfs clients read node-local data from the server thread which is a bottleneck. In the case of Laminated files, we can fetch the node-local extents from the server and perform read locally. With the new proposed changes
- The client has a new configuration for node_local_extents
- On the first read of a laminated file if client.node_local_extents is enabled we fetch extents from the server and update the local extents
- This allows reads to be executed locally
- on server we get our extents using unifyfs_invoke_find_extents_rpc
In these cases, we do only one RPC per request group and fetch extents for all laminated files.
Motivation and Context
This feature would enable faster reads by clients in the case of laminated files. Some examples of this are,
- read-only workload where data is cached in unifyfs node-locally.
- producer/consumer use cases like MuMMI are colocated in the same node.
How Has This Been Tested?
In the API test case, I have enabled node_local_extents to true. This works with existing tests well as for non-laminated files this optimization is not applied. Whereas for laminated files it is.
Types of changes
- [ ] Bug fix (non-breaking change which fixes an issue)
- [X] New feature (non-breaking change which adds functionality)
- [X] Performance enhancement (non-breaking change which improves efficiency)
- [ ] Code cleanup (non-breaking change which makes code smaller or more readable)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Testing (addition of new tests or update to current tests)
- [ ] Documentation (a change to man pages or other documentation)
Checklist:
- [X] My code follows the UnifyFS code style requirements.
- [x] I have updated the documentation accordingly.
- [X] I have read the CONTRIBUTING document.
- [X] I have added tests to cover my changes.
- [X] All new and existing tests passed.
- [X] All commit messages are properly formatted.
@hariharan-devarajan Thanks, this is getting really close to being ready - just a few more requested changes. After that, we'll need to rebase/squash down to a single commit.
I have rebased it :)