brimdata
Optimized pool key predicates
This pr introduces new functionality that uses pool key predicates in the pushdown to optimize both object and object seek selection. Both objects and their seek offsets are selected by using augmented version of the filter used in the original query. Optimized pool key lookups are limited in the following ways:
- Will only work on the basic comparison operators "==", ">=", ">", "<=", "<"
- The right hand side must be a literal.
- Will not work on expressions nested under unary operators.
This pr also removes the from pool range syntax that was the former approach to limiting the scope of a query.
Fixes #3745
Still pending before I take this out of draft mode:
- [x] Write a more comprehensive test suite for these changes
- [x] Update documentation- remove references to range syntax and add documentation about pool key predicates are optimized
- [x] Fix issue with parentheses in zfmt
- [x] There's a couple places in the code where the logic isn't immediately obvious- add some comments
In a team sync, @mattnibs confirmed that this filtering needs to happen in a single pipeline element to be recognized as optimize-able. I've opened https://github.com/brimdata/brim/issues/2449 to remind us to change how the app crafts queries in order to take advantage of this optimization when it lands.
