iceberg
iceberg copied to clipboard
Spark: Support rewrite file with z-order for nested Struct type
Notes
This change adds support for rewriting files with Z-ordering for nested Struct types in Iceberg. This allows more efficient data organization and retrieval in scenarios where nested Struct types are present.
The implementation involves extending the existing retrieve z-order columns logic to handle nested Struct types.
Testing
Unit tests
Tested real dataset on EMR 7.0.0, and verified the data was ordered based on nested column.