rtpsw
rtpsw
This is a feature I intend to implement once the design discussion reaches agreement. cc @cpcloud, @icexelloss
> I suspect that we want to have a generic table UDF mechanism and not have something specific for data sources. Do you mean to extend the `ibis.table` API to...
@icexelloss, my main goal in this issue is to figure out a design that fits Ibis. I think it would be easier to present an end-to-end (Ibis/Ibis-Substrait-Arrow) design following this....
After some experimentation, I'd propose the Ibis/Ibis-Substrait design shown below. @cpcloud, could you provide some feedback? In Ibis: - Add a new `UserDefinedSourceTable` operation derived from `PhysicalTable` that has similar...
The proposed design can be seen [here](https://github.com/rtpsw/ibis/compare/0eb9bc50..93ac74c9#diff-8f6e03b936fae42459cef36d6bb2691392fb1ed9bd03c9846152ac6ebb6da892R5267-R5284) and [here](https://github.com/rtpsw/ibis-substrait/compare/2e93a1d8..7ea2888d#diff-14bbad08fcc797e6913a5f446547e349f60aefe5b2ede385c32c87826baabdbaR658-R677). If this design is acceptable, I'll set up a corresponding PR.
@cpcloud, what are your thoughts about this?
@pitrou, could you review? cc @icexelloss
I developed this PR's code while experimenting with designs for AsOfJoin; however, the design I chose to proceed with does not need this PR's code. So, I'm fine if this...
> @rtpsw Do we need this for the AsOfJoin improvement? No.
Just to clarify, I'm holding on looking into specific comments on this PR until and if we decide to move forward with it.