gobblin
gobblin copied to clipboard
[GOBBLIN-1714] Prevent silent failure of data copy if files don't exist in source
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
- [ ] My PR addresses the following Gobblin JIRA issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR"
- https://issues.apache.org/jira/browse/GOBBLIN-1714
Description
- [ ] Here are some details about my PR, including screenshots (if applicable):
- [ ]
- [ ] Throws exception if no files found in source for data copy
- [ ] Doesn't throw exception if no files found in target for data copy as the files will be created
Tests
- [ ] My PR adds the following unit tests OR does not need testing for this extremely good reason:
- [ ] All current tests pass
Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
- Subject is separated from body by a blank line
- Subject is limited to 50 characters
- Subject does not end with a period
- Subject uses the imperative mood ("add", not "adding")
- Body wraps at 72 characters
- Body explains "what" and "why", not "how"
Codecov Report
Merging #3568 (71161bf) into master (78484e2) will decrease coverage by
0.14%. The diff coverage is0.00%.
@@ Coverage Diff @@
## master #3568 +/- ##
============================================
- Coverage 46.93% 46.78% -0.15%
+ Complexity 10580 10567 -13
============================================
Files 2105 2111 +6
Lines 82149 82468 +319
Branches 9147 9174 +27
============================================
+ Hits 38558 38586 +28
- Misses 40050 40323 +273
- Partials 3541 3559 +18
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...data/management/copy/RecursiveCopyableDataset.java | 86.36% <0.00%> (ø) |
|
| ...blin/data/management/copy/RecursivePathFinder.java | 0.00% <0.00%> (-92.86%) |
:arrow_down: |
| ...data/management/copy/extractor/EmptyExtractor.java | 0.00% <0.00%> (-83.34%) |
:arrow_down: |
| ...a/management/copy/hive/HiveLocationDescriptor.java | 36.17% <0.00%> (-14.90%) |
:arrow_down: |
| ...anagement/copy/hive/UnpartitionedTableFileSet.java | 66.66% <0.00%> (-12.97%) |
:arrow_down: |
| ...ata/management/copy/hive/HiveCopyEntityHelper.java | 51.41% <0.00%> (-11.59%) |
:arrow_down: |
| ...ache/gobblin/util/commit/DeleteFileCommitStep.java | 34.04% <0.00%> (-8.52%) |
:arrow_down: |
| ...a/management/copy/publisher/CopyDataPublisher.java | 67.54% <0.00%> (-7.95%) |
:arrow_down: |
| .../hive/filter/LookbackPartitionFilterGenerator.java | 78.94% <0.00%> (-5.27%) |
:arrow_down: |
| ...ervice/modules/core/GobblinServiceGuiceModule.java | 84.74% <0.00%> (-5.26%) |
:arrow_down: |
| ... and 24 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more