spark icon indicating copy to clipboard operation
spark copied to clipboard

[WIP][SPARK-48157][SQL] Add collation support for CSV expressions

Open uros-db opened this issue 1 year ago • 0 comments

What changes were proposed in this pull request?

Introduce collation awareness for CSV expressions: from_csv, schema_of_csv, to_csv.

Why are the changes needed?

Add collation support for CSV expressions in Spark.

Does this PR introduce any user-facing change?

Yes, users should now be able to use collated strings within arguments for CSV functions: from_csv, schema_of_csv, to_csv.

How was this patch tested?

E2e sql tests.

Was this patch authored or co-authored using generative AI tooling?

No.

uros-db avatar May 09 '24 09:05 uros-db