[BUG] array node sub node control flow inheritance
Tracking issue
Why are the changes needed?
Array node increments workflow parallelism from the parent node. Right now, the same control flow that tracks parallelism for the workflow is passed down to subnodes allowing subnodes to bump the workflow parallelism which could lead to a double counting of nodes getting evaluated in a workflow.
If/when ArrayNode supports subworkflows or dynamics then we can pass back in the parent control flow.
What changes were proposed in this pull request?
Init a new control flow when creating the exec context for array node subnodes
How was this patch tested?
Ran a few workflows locally
Setup process
Screenshots
Check all the applicable boxes
- [ ] I updated the documentation accordingly.
- [x] All new and existing tests passed.
- [x] All commits are signed-off.
Related PRs
Docs link
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 61.01%. Comparing base (
8805613) to head (a00afea). Report is 447 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #5505 +/- ##
==========================================
+ Coverage 60.15% 61.01% +0.85%
==========================================
Files 646 794 +148
Lines 45883 51445 +5562
==========================================
+ Hits 27603 31389 +3786
- Misses 15672 17164 +1492
- Partials 2608 2892 +284
| Flag | Coverage Δ | |
|---|---|---|
| unittests-datacatalog | 69.31% <ø> (ø) |
|
| unittests-flyteadmin | 58.73% <ø> (ø) |
|
| unittests-flytecopilot | 17.79% <ø> (ø) |
|
| unittests-flytectl | 68.03% <ø> (?) |
|
| unittests-flyteidl | 79.04% <ø> (ø) |
|
| unittests-flyteplugins | 61.85% <ø> (ø) |
|
| unittests-flytepropeller | 57.31% <100.00%> (+<0.01%) |
:arrow_up: |
| unittests-flytestdlib | 65.82% <ø> (+0.02%) |
:arrow_up: |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
@hamersaw This would mess with the node + task count. I'll look back into this.
Cleaning stale PRs. Please reopen if you wan to discuss this further.