pinot icon indicating copy to clipboard operation
pinot copied to clipboard

add SegmentContext to collect validDocIds bitmaps for many segments together

Open klsince opened this issue 1 year ago • 1 comments

This PR adds SegmentContext used to collect segment related context before query execution. E.g. this makes it easier to collect validDocIds for many segments together, so that we can add the support for consistent table view for upsert table easier later on.

This PR didn't change current functionalities but moved the logic of getting validDocId bitmaps upto one place, i.e. tableDataMgr.getSegmentContexts(selectedSegments).

klsince avatar Mar 21 '24 23:03 klsince

Codecov Report

Attention: Patch coverage is 82.99320% with 25 lines in your changes are missing coverage. Please review.

Project coverage is 62.04%. Comparing base (59551e4) to head (a13897e). Report is 244 commits behind head on master.

Files Patch % Lines
...cal/upsert/BasePartitionUpsertMetadataManager.java 56.25% 14 Missing :warning:
...ata/manager/realtime/RealtimeTableDataManager.java 0.00% 5 Missing :warning:
...psert/ConcurrentMapTableUpsertMetadataManager.java 0.00% 3 Missing :warning:
...core/query/executor/ServerQueryExecutorV1Impl.java 92.85% 1 Missing :warning:
...t/ConcurrentMapPartitionUpsertMetadataManager.java 50.00% 0 Missing and 1 partial :warning:
...gment/local/upsert/TableUpsertMetadataManager.java 0.00% 1 Missing :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #12694      +/-   ##
============================================
+ Coverage     61.75%   62.04%   +0.28%     
+ Complexity      207      198       -9     
============================================
  Files          2436     2464      +28     
  Lines        133233   134762    +1529     
  Branches      20636    20813     +177     
============================================
+ Hits          82274    83607    +1333     
- Misses        44911    45002      +91     
- Partials       6048     6153     +105     
Flag Coverage Δ
custom-integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration <0.01% <0.00%> (-0.01%) :arrow_down:
integration1 <0.01% <0.00%> (-0.01%) :arrow_down:
integration2 0.00% <0.00%> (ø)
java-11 34.95% <70.06%> (-26.76%) :arrow_down:
java-21 61.92% <82.99%> (+0.30%) :arrow_up:
skip-bytebuffers-false 62.00% <82.99%> (+0.25%) :arrow_up:
skip-bytebuffers-true 61.86% <82.99%> (+34.14%) :arrow_up:
temurin 62.04% <82.99%> (+0.28%) :arrow_up:
unittests 62.03% <82.99%> (+0.28%) :arrow_up:
unittests1 46.69% <70.06%> (-0.20%) :arrow_down:
unittests2 27.99% <12.92%> (+0.26%) :arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar Mar 21 '24 23:03 codecov-commenter