gobblin
gobblin copied to clipboard
[GOBBLIN-1703] avoid double quota increase for adhoc flows
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
- [X] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1703
Description
- [X] Here are some details about my PR, including screenshots (if applicable): When a gaas flow request comes, resource handler checks the quota right there. However, if the flow has runImmediately=true, the quota will be checked again when the first job starts. This should be avoided.
Tests
- [ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:
Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
- Subject is separated from body by a blank line
- Subject is limited to 50 characters
- Subject does not end with a period
- Subject uses the imperative mood ("add", not "adding")
- Body wraps at 72 characters
- Body explains "what" and "why", not "how"
Codecov Report
Merging #3550 (30049f6) into master (ae2ea19) will decrease coverage by
0.89%. The diff coverage is46.50%.
@@ Coverage Diff @@
## master #3550 +/- ##
============================================
- Coverage 47.76% 46.86% -0.90%
- Complexity 8557 10617 +2060
============================================
Files 1705 2111 +406
Lines 64976 82667 +17691
Branches 7036 9202 +2166
============================================
+ Hits 31035 38742 +7707
- Misses 31240 40363 +9123
- Partials 2701 3562 +861
| Impacted Files | Coverage Δ | |
|---|---|---|
| .../org/apache/gobblin/service/ServiceConfigKeys.java | 0.00% <ø> (ø) |
|
| ...gobblin/runtime/spec_store/MysqlBaseSpecStore.java | 89.16% <ø> (ø) |
|
| ...lin/service/modules/flow/MultiHopFlowCompiler.java | 67.25% <0.00%> (-1.62%) |
:arrow_down: |
| ...odules/orchestration/AbstractUserQuotaManager.java | 100.00% <ø> (+22.85%) |
:arrow_up: |
| ...ervice/modules/flow/BaseFlowToJobSpecCompiler.java | 60.68% <12.50%> (-3.54%) |
:arrow_down: |
| ...e/modules/orchestration/MysqlUserQuotaManager.java | 47.34% <31.38%> (-26.53%) |
:arrow_down: |
| ...blin/service/modules/orchestration/DagManager.java | 82.09% <50.00%> (-0.14%) |
:arrow_down: |
| ...odules/orchestration/InMemoryUserQuotaManager.java | 70.07% <73.03%> (+5.97%) |
:arrow_up: |
| .../modules/scheduler/GobblinServiceJobScheduler.java | 64.62% <100.00%> (ø) |
|
| ...e/gobblin/service/monitoring/GitConfigMonitor.java | 95.23% <0.00%> (-4.77%) |
:arrow_down: |
| ... and 412 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
Functionality of checking quota for every job is left unchanged that can be done in the other PR. Handling parallel flow functionality is also left unchanged for the other PR.