CrayLabs
Spike: Python Decorators for SmartRedis
Description
Python decorators can be used to add functionality to existing workflows with minimal code changes. One idea submitted by a potential SmartSim user is to provide a cache decorator which would essentially allow the result of a Python evaluation to be stored into the database. There are likely a number of similar types of operations that we could likely provide.
Justification
Both users with existing Python workflows or those creating new ones can take advantage of the datastore provided by the orchestrator in a potentially simpler/cleaner way and can also help with key management. For example, with the aforementioned cache decorator, one could quickly wrap a numpy or scipy function to put the result in the database. Similar decorators might also simplify the creation/transformation of SmartRedis datasets (for example, if the metadata is known a priori).
Implementation Strategy
- A survey should be done of users and developers to understand what decorators would be useful broadly across workflows
- Capture any performance or side-effects that might arise from the use of these decorators (e.g. inadvertent overwrites of keys, maybe allow for asynchronous communication with the database)
- Implement a subset of the decorators
