[GOBBLIN-1552] determine flow status based on the fact if dag manager is enabled

[arjun4084346]

Dear Gobblin maintainers,

Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!

JIRA

• [x] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
  • https://issues.apache.org/jira/browse/GOBBLIN-1552

Description

• [x] Here are some details about my PR, including screenshots (if applicable): flow status is determined by looking at the flow level events. but flow level events are not emitted outside of dag manager. so currently flow status is not being determined correctly when dag manager is disabled. this pr adds a field flowStatus to FlowStatus class, which is determined by JobStatus'es when DagManager is not enabled. in a way it used to be done prior to 59f3bee#diff-141f30f7aa658e9e18316e3e5ecea3b92865dcc88a4c0809b6d5c0ef1598e410

Another refactoring this PR does is to make getJobStatusesForFlowExecution in jobStatusRetriever return a list instead of an Iterator, this is done because flow status determination needs to iterate through this iterator and the iterator cannot be re-parsed so it become unusable afterwards.

Tests

• [x] My PR adds the following unit tests OR does not need testing for this extremely good reason: added a couple of unit tests and updated the existing ones

Commits

• [x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
  1. Subject is separated from body by a blank line
  2. Subject is limited to 50 characters
  3. Subject does not end with a period
  4. Subject uses the imperative mood ("add", not "adding")
  5. Body wraps at 72 characters
  6. Body explains "what" and "why", not "how"

[codecov-commenter]

Codecov Report

Merging #3402 (be7a4c8) into master (c05416e) will decrease coverage by 0.02%. The diff coverage is 29.87%.

@@             Coverage Diff              @@
##             master    #3402      +/-   ##
============================================
- Coverage     46.53%   46.51%   -0.03%     
+ Complexity    10243    10242       -1     
============================================
  Files          2062     2063       +1     
  Lines         80431    80475      +44     
  Branches       8984     8993       +9     
============================================
- Hits          37432    37429       -3     
- Misses        39534    39579      +45     
- Partials       3465     3467       +2     

Impacted Files	Coverage Δ
.../org/apache/gobblin/service/ServiceConfigKeys.java	0.00% <ø> (ø)
...gobblin/service/monitoring/JobStatusRetriever.java	1.04% <0.00%> (-0.35%)	:arrow_down:
.../service/monitoring/LocalFsJobStatusRetriever.java	0.00% <0.00%> (ø)
...lin/service/FlowExecutionResourceLocalHandler.java	5.20% <20.00%> (-0.92%)	:arrow_down:
.../apache/gobblin/service/monitoring/FlowStatus.java	71.42% <50.00%> (-11.91%)	:arrow_down:
...bblin/service/monitoring/FsJobStatusRetriever.java	55.17% <50.00%> (ø)
...obblin/service/monitoring/FlowStatusGenerator.java	59.09% <73.33%> (+3.53%)	:arrow_up:
...vice/modules/core/GobblinServiceConfiguration.java	93.33% <100.00%> (ø)
...blin/service/modules/orchestration/DagManager.java	72.96% <100.00%> (ø)
...in/service/monitoring/MysqlJobStatusRetriever.java	92.85% <100.00%> (ø)
... and 9 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c05416e...be7a4c8. Read the comment docs.