aqueduct
aqueduct copied to clipboard
Implement non-tabular type support and eager execution
Describe your changes and why you are making these changes
This PR:
- Expands our type support beyond table, float, and bool.
- Changes the execution mode to be eager. We will add the lazy mode as a follow-up.
A couple things still need to happen before this PR can be merged:
- [ ] Add logic to the UI such that when the artifact type is one of the new types, we show a message to the user that previewing of these new types aren't supported yet. (we will implement it in follow-up PRs).
- [ ] Add a data migration step to migrate the original
artifact.speccolumn toartifact.typeso that existing workflows continue to work.
This is a large PR, but a lot of it is just mechanical changes and adjusting the test to cope with the eager execution mode. Let's set up a meeting to go over the important changes first.
Related issue number (if any)
Checklist before requesting a review
- [x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
- [x] I have performed a self-review of my code.
- [ ] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
- [ ] If this is a new feature, I have added unit tests and integration tests.
- [x] I have run the integration tests locally and they are passing.
- [x] I have run the linter script locally (See
python3 scripts/run_linters.py -hfor usage). - [x] All features on the UI continue to work correctly.
- [ ] Added one of the following CI labels:
-
run_integration_test: Runs integration tests -
skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)
-
@kenxu95 addressed comments and added integration tests for the type hierarchy restrictions.