spark
spark copied to clipboard
[SPARK-54625][SQL][4.0] UTF8String#reverse should check offset and length on copying
What changes were proposed in this pull request?
This PR backports #53366 to branch-4.0.
This PR aims to check offset and length on copying in UTF8String#reverse.
For details, see https://lists.apache.org/thread/d9pvkh3jbsq8lc33v75kmwq5wg57422h (Only PMC members can read with login).
To avoid performance regression, this PR choose to check offset and length rather than validate the input UTF-8 string.
Why are the changes needed?
For safety.
Does this PR introduce any user-facing change?
Yes, but doesn't break compatibility.
How was this patch tested?
Example queries mentioned in this thread works even though the results are broken.
All the operation defined in UTF8String are expected to work correctly with valid UTF-8 strings so the broken results with invalid UTF-8 strings should be reasonable.
Was this patch authored or co-authored using generative AI tooling?
No.