spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-48619] Fix reference error when grouping by MapType

Open stevomitric opened this issue 1 year ago • 2 comments

What changes were proposed in this pull request?

In this PR, rule InsertMapSortInGroupingExpressions was modified to add an alias for map_sort(col) . Grouping and aggregate expressions are then modified to use this alias.

Why are the changes needed?

Currently, trying to reference a map column that is being grouped by, causes reference errors because we are only replacing the grouping expression m with map_sort(m). The following query fails:

SELECT m FROM table GROUP BY m

Does this PR introduce any user-facing change?

No

How was this patch tested?

New Tests in this PR, and existing tests regarding grouping of map types.

Was this patch authored or co-authored using generative AI tooling?

No

stevomitric avatar Jun 13 '24 12:06 stevomitric

@n-young-db @cloud-fan can you please take a look at this.

stevomitric avatar Jun 13 '24 12:06 stevomitric

Approved but please address comments! Just didn't want to block over the weekend.

n-young-db avatar Jun 14 '24 16:06 n-young-db

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

github-actions[bot] avatar Sep 28 '24 00:09 github-actions[bot]