aqueduct icon indicating copy to clipboard operation
aqueduct copied to clipboard

Implement non-tabular type support and eager execution

Open cw75 opened this issue 3 years ago • 1 comments

Describe your changes and why you are making these changes

This PR:

  1. Expands our type support beyond table, float, and bool.
  2. Changes the execution mode to be eager. We will add the lazy mode as a follow-up.

A couple things still need to happen before this PR can be merged:

  • [ ] Add logic to the UI such that when the artifact type is one of the new types, we show a message to the user that previewing of these new types aren't supported yet. (we will implement it in follow-up PRs).
  • [ ] Add a data migration step to migrate the original artifact.spec column to artifact.type so that existing workflows continue to work.

This is a large PR, but a lot of it is just mechanical changes and adjusting the test to cope with the eager execution mode. Let's set up a meeting to go over the important changes first.

Related issue number (if any)

Checklist before requesting a review

  • [x] I have created a descriptive PR title. The PR title should complete the sentence "This PR...".
  • [x] I have performed a self-review of my code.
  • [ ] I have included a small demo of the changes. For the UI, this would be a screenshot or a Loom video.
  • [ ] If this is a new feature, I have added unit tests and integration tests.
  • [x] I have run the integration tests locally and they are passing.
  • [x] I have run the linter script locally (See python3 scripts/run_linters.py -h for usage).
  • [x] All features on the UI continue to work correctly.
  • [ ] Added one of the following CI labels:
    • run_integration_test: Runs integration tests
    • skip_integration_test: Skips integration tests (Should be used when changes are ONLY documentation/UI)

cw75 avatar Aug 12 '22 07:08 cw75

@kenxu95 addressed comments and added integration tests for the type hierarchy restrictions.

cw75 avatar Aug 16 '22 21:08 cw75