spark
spark copied to clipboard
[SPARK-47988][SQL] When the collationId is invalid, throw `COLLATION_INVALID_ID`
What changes were proposed in this pull request?
The pr aims to
- convert
ArrayIndexOutOfBoundsExceptiontoSparkArrayIndexOutOfBoundsException. - refactor
CollationFactoryto extract some of the same logic into a method, reduce redundancy. - when calling
String#toUpperCase()with parametersLocale.ROOT, which conforms to the traditional rules of Spark, as follows: https://github.com/apache/spark/blob/master/scalastyle-config.xml#L247-L258
Why are the changes needed?
Before:
After:
Does this PR introduce any user-facing change?
Yes, When the collationId is invalid, the user will get a more friendly error prompt.
How was this patch tested?
- Add new UT.
- Pass GA.
Was this patch authored or co-authored using generative AI tooling?
No.
cc @uros-db @cloud-fan @MaxGekk
i.e. we are working to remove the CollationTable concept entirely, as we expand collation support
I think this will soon no longer be relevant, since we are re-implementing how collationIds work in Spark: #46180
Okay, let me have a look.