feat: correlation support
Which issue does this PR close?
Supports CORR
The implementation mostly is the same as the DataFusion's implementation. The reason
we have our own implementation is that DataFusion has UInt64 for state_field count,
while Spark has Double for count. Also adding null_on_divide_by_zero
to be consistent with Spark's implementation.
Closes #.
Rationale for this change
What changes are included in this PR?
How are these changes tested?
Codecov Report
Attention: Patch coverage is 85.71429% with 2 lines in your changes are missing coverage. Please review.
Project coverage is 34.02%. Comparing base (
14494d3) to head (fead995). Report is 26 commits behind head on main.
| Files | Patch % | Lines |
|---|---|---|
| .../scala/org/apache/comet/serde/QueryPlanSerde.scala | 85.71% | 1 Missing and 1 partial :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #456 +/- ##
============================================
- Coverage 34.02% 34.02% -0.01%
- Complexity 857 859 +2
============================================
Files 116 116
Lines 38565 38671 +106
Branches 8517 8564 +47
============================================
+ Hits 13120 13156 +36
- Misses 22691 22753 +62
- Partials 2754 2762 +8
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
cc @andygrove @viirya
Merged. Thanks @huaxingao @andygrove @kazuyukitanimura
Thanks @viirya @andygrove @kazuyukitanimura