hive icon indicating copy to clipboard operation
hive copied to clipboard

HIVE-28537: Iceberg: allow only partition columns in the WHERE clause

Open zratkai opened this issue 1 year ago • 4 comments

Change-Id: Ic85efd70599413cdb96073c6cb50690fbc1c11b0

What changes were proposed in this pull request?

Limiting the columns in where clause in compaction to partition columns.

Why are the changes needed?

Currently Hive Iceberg compaction supports any columns in the WHERE clause predicate. However, compaction happens only on partition level. Performance can be improved if the columns in WHERE clause predicate will be limited to partition columns.

Does this PR introduce any user-facing change?

Yes, user will get an exception if uses non partition columns in alter table compaction query.

Is the change a dependency upgrade?

No.

How was this patch tested?

With qtest.

zratkai avatar Oct 03 '24 13:10 zratkai

+1, Looks good to me.

Added a few minor formatting comments. There is also a q-test failure iceberg_major_compaction_partition_evolution_w_id_spec_w_filter The test needs to be adjusted for these code changes.

difin avatar Oct 03 '24 18:10 difin

please address only relevant items from https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=5483&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true

deniskuzZ avatar Oct 18 '24 08:10 deniskuzZ

@deniskuzZ thanks for the review! Please check the modifications!

zratkai avatar Oct 18 '24 15:10 zratkai