datafusion
datafusion copied to clipboard
apache
Dynamic filter pushdown to probe side
Which issue does this PR close?
Closes #7955
Rationale for this change
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?
still in draft, similar to DuckDB what does, support single column less than and greater than to push to probe side. Gonna add benchmark test and more test cases later this week.
When I was implementing this , the hard part is how to collect global build side information in partition-mode, currently my idea is to using a lock to ensure all the build-side operations are done before any probe-phase operations to start. Don't know if it is a good choice.