facebook
Implement Interactive Loop with candidate pregeneration
Summary: In this diff we create an interactive loop API (based loosely off the existing Loop API) in order to facilitate candidate pregeneration. This can be useful in many contexts, espcially interactive experiments in which trial evaluation entails eliciting a response from a real human. Candidate pregeneration uses the time waiting for trail evaluation to generate new candidates from the data available. Note that this is a tradeoff -- a larger lookahead will result in more trials being generated with less data compared to a smaller lookahead (or no lookahead as in conventional Ax usage) and should only be used in contexts where it is necessary for the user to not experience any "lag" while candidates are being generated.
Reviewed By: lena-kashtelyan
Differential Revision: D39505104
Codecov Report
Base: 94.85% // Head: 94.85% // Increases project coverage by +0.00% :tada:
Coverage data is based on head (
6a8c0b7) compared to base (1613482). Patch coverage: 96.59% of modified lines in pull request are covered.
:exclamation: Current head 6a8c0b7 differs from pull request most recent head 9bb35b2. Consider uploading reports for the commit 9bb35b2 to get more accurate results
Additional details and impacted files
@@ Coverage Diff @@
## main #1144 +/- ##
=======================================
Coverage 94.85% 94.85%
=======================================
Files 412 414 +2
Lines 36954 37039 +85
=======================================
+ Hits 35053 35134 +81
- Misses 1901 1905 +4
| Impacted Files | Coverage Δ | |
|---|---|---|
| ax/utils/tutorials/cnn_utils.py | 0.00% <0.00%> (ø) |
|
| ax/service/interactive_loop.py | 100.00% <100.00%> (ø) |
|
| ax/service/tests/test_interactive_loop.py | 100.00% <100.00%> (ø) |
|
| ax/utils/common/testutils.py | 86.16% <0.00%> (-1.26%) |
:arrow_down: |
| ax/service/scheduler.py | 96.01% <0.00%> (-0.45%) |
:arrow_down: |
| ax/modelbridge/tests/test_registry.py | 99.51% <0.00%> (+0.48%) |
:arrow_up: |
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.
