datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

fix: Support Substrait's compound names also for window functions

Open Blizzara opened this issue 1 year ago • 1 comments

Which issue does this PR close?

#10653 fixed the name mismatch (substrait's sum:int32 vs DF's sum) for scalar functions, and #10842 fixed it for aggregates. This fixes it for windows as well.

Closes #.

Rationale for this change

What changes are included in this PR?

Substrait's standard defines function names to be compound signatures including the arg types the function takes. DataFusion doesn't yet produce names that'd correspond to it, but we should still be able to consume those names. That's already been fixed for scalar and aggregate functions, this fixes it for window functions as well.

Also, I took the liberty to refactor/harmonize/clean up the function handling across scalar/aggregate/window. There should be no functional changes, but hopefully the code is a bit easier to read now.

Are these changes tested?

Tested with existing unit tests

Are there any user-facing changes?

Fixes a typo in one public function, though I don't expect anyone to be using that outside of consumer.rs. from_substriat_func_args -> from_substrait_func_args

Blizzara avatar Jun 28 '24 13:06 Blizzara

Could you please add some test coverage (or perhaps correct me if I missed it in this PR)

That was indeed missing, I had planned to add it but forgot 😅 Added in https://github.com/apache/datafusion/pull/11163/commits/a08e62b0d2c4041c3c5d1e78ad2ab452c9d745a5, that also surfaced another issue in consuming bound type which I fixed as well

Blizzara avatar Jul 01 '24 10:07 Blizzara

Thanks @Blizzara

alamb avatar Jul 01 '24 13:07 alamb