Peter Toth

[SPARK-39887][SQL] RemoveRedundantAliases should keep aliases that make the output of projection nodes unique

5

### What changes were proposed in this pull request? Keep the output attributes of a `Union` node's first child in the `RemoveRedundantAliases` rule to avoid correctness issues. ### Why are...

[LIVY-694] Upgrade Jetty to 9.4.18.v20190429

6

## What changes were proposed in this pull request? Upgrade Jetty to 9.4.18.v20190429 to fix CVEs. ## How was this patch tested? Existing UTs.

[SPARK-38717][SQL] Handle Hive's bucket spec case preserving behaviour

6

### What changes were proposed in this pull request? When converting a native table metadata representation `CatalogTable` to `HiveTable` make sure bucket spec uses an existing column. ### Does this...

[SPARK-40086][SQL] Improve AliasAwareOutputPartitioning to take all aliases into account

2

### What changes were proposed in this pull request? Currently `AliasAwareOutputPartitioning` takes only the last alias by aliased expressions into account. We could avoid more shuffles with better alias handling....

[SPARK-40259][SQL] Support Parquet DSv2 in subquery plan merge

1

### What changes were proposed in this pull request? After https://github.com/apache/spark/pull/32298 we were able to merge scalar subquery plans, but DSv2 sources couldn't benefit from that improvement. The reason for...

Fix TreeNode transform and rewrite APIs

67

## Which issue does this PR close? Part of https://github.com/apache/arrow-datafusion/issues/8913 ## Rationale for this change 1. The `TreeNode.rewrite()` / `TreeNodeRewriter` is very inconsistent with other `TreeNode` apply / visit functions:...

[SPARK-24497][SQL] Support recursive SQL

16

### What changes were proposed in this pull request? This PR adds recursive query feature to Spark SQL. A recursive query is defined using the `WITH RECURSIVE` keywords and referring...

[SPARK-40193][SQL] Merge subquery plans with different filters

15

### What changes were proposed in this pull request? After https://github.com/apache/spark/pull/32298 we were able to merge scalar subquery plans. This PR is a follow-up improvement to the merging logic to...

Add reference visitor `TreeNode` APIs

13

## Which issue does this PR close? Part of https://github.com/apache/datafusion/issues/10121, required for https://github.com/apache/datafusion/issues/10505 and https://github.com/apache/datafusion/issues/10426. ## Rationale for this change The current `TreeNode` visitor APIs (`TreeNode::visit()` and `TreeNode::apply()`) have a...

Better CSE identifier

2

This is a draft PR that implements the ideas from https://github.com/apache/datafusion/issues/10426#issuecomment-2105664520. ## Which issue does this PR close? Closes https://github.com/apache/datafusion/issues/10426. ## Rationale for this change ## What changes are included...