Add flag to allow skipping decoder exceptions in LLRealtimeSegmentDataManager #9045

[ddcprg]

Helps fixing #9045

This flag can be used to control whether exceptions thrown by the decoder should be ignore. For example, at the moment the CSV decoder throw an exception if the payload cannot be decoded, this stops consumption. With this flag users may decide to ignore these errors and continue consumption.

Exception will be retied by the consumption loop until the maximum number of retires is reached.

Decoders internally may decide to retry themselves when an exception is thrown and bubble up the exception as a last resort

[codecov-commenter]

Codecov Report

Merging #9163 (cfa43f4) into master (e8a8684) will decrease coverage by 35.13%. The diff coverage is 17.39%.

@@              Coverage Diff              @@
##             master    #9163       +/-   ##
=============================================
- Coverage     69.99%   34.86%   -35.14%     
+ Complexity     4682      194     -4488     
=============================================
  Files          1848     1848               
  Lines         98572    98588       +16     
  Branches      14967    14968        +1     
=============================================
- Hits          68999    34368    -34631     
- Misses        24723    61364    +36641     
+ Partials       4850     2856     -1994     

Flag	Coverage Δ
integration1	?
integration2	24.90% <17.39%> (+0.08%)	:arrow_up:
unittests1	?
unittests2	15.34% <0.00%> (-0.06%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...java/org/apache/pinot/spi/stream/StreamConfig.java	0.00% <0.00%> (-88.27%)	:arrow_down:
...pache/pinot/spi/stream/StreamConfigProperties.java	0.00% <ø> (-100.00%)	:arrow_down:
...manager/realtime/LLRealtimeSegmentDataManager.java	57.40% <40.00%> (-12.95%)	:arrow_down:
...in/java/org/apache/pinot/spi/utils/BytesUtils.java	0.00% <0.00%> (-100.00%)	:arrow_down:
.../java/org/apache/pinot/spi/utils/BooleanUtils.java	0.00% <0.00%> (-100.00%)	:arrow_down:
...java/org/apache/pinot/spi/trace/BaseRecording.java	0.00% <0.00%> (-100.00%)	:arrow_down:
...java/org/apache/pinot/spi/trace/NoOpRecording.java	0.00% <0.00%> (-100.00%)	:arrow_down:
...ava/org/apache/pinot/spi/config/table/FSTType.java	0.00% <0.00%> (-100.00%)	:arrow_down:
...ava/org/apache/pinot/spi/config/user/RoleType.java	0.00% <0.00%> (-100.00%)	:arrow_down:
...ava/org/apache/pinot/spi/data/MetricFieldSpec.java	0.00% <0.00%> (-100.00%)	:arrow_down:
... and 1081 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more