pinot
pinot copied to clipboard
Handle exception in realtime decoder gracefully
This PR continues the effort from #9163
We already added continueOnError flag in previous PRs to handle errors gracefully. This PR extends the same configs to handle decoder exceptions as well.
A new config flag has also been added called skipPartialRecords through which user can decided whether to consume these partial records or not after continueOnError.
Codecov Report
Merging #9511 (3158047) into master (91fa86b) will decrease coverage by
0.03%. The diff coverage is13.33%.
@@ Coverage Diff @@
## master #9511 +/- ##
============================================
- Coverage 28.13% 28.09% -0.04%
Complexity 53 53
============================================
Files 1936 1936
Lines 103927 103961 +34
Branches 15770 15778 +8
============================================
- Hits 29235 29211 -24
- Misses 71832 71901 +69
+ Partials 2860 2849 -11
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | 25.35% <5.00%> (-0.06%) |
:arrow_down: |
| integration2 | 24.43% <13.33%> (-0.26%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Impacted Files | Coverage Δ | |
|---|---|---|
| .../local/recordtransformer/CompositeTransformer.java | 0.00% <ø> (ø) |
|
| ...cal/recordtransformer/SanitizationTransformer.java | 0.00% <0.00%> (ø) |
|
| ...gment/local/segment/creator/TransformPipeline.java | 0.00% <0.00%> (ø) |
|
| ...t/creator/impl/SegmentIndexCreationDriverImpl.java | 0.00% <0.00%> (ø) |
|
| ...ache/pinot/segment/local/utils/IngestionUtils.java | 0.00% <0.00%> (ø) |
|
| ...ot/spi/config/table/ingestion/IngestionConfig.java | 0.00% <0.00%> (ø) |
|
| .../core/segment/processing/mapper/SegmentMapper.java | 87.87% <25.00%> (-1.19%) |
:arrow_down: |
| ...manager/realtime/HLRealtimeSegmentDataManager.java | 78.08% <50.00%> (-2.20%) |
:arrow_down: |
| ...manager/realtime/LLRealtimeSegmentDataManager.java | 61.64% <54.54%> (+2.19%) |
:arrow_up: |
| ...ntroller/helix/core/minion/TaskMetricsEmitter.java | 34.88% <0.00%> (-51.17%) |
:arrow_down: |
| ... and 42 more |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more