HIVE-28676: Support custom partition patterns in MSCK repair table

[VenkatSNarayanan]

Adds support for custom partitioning patterns to MSCK repair table.

What changes were proposed in this pull request?

Adds support for custom partitioning patterns to MSCK repair table.

Why are the changes needed?

HCatStorer supports custom partitioning patterns when using dynamic partitioning, but Hive itself does not support this. This change adds support for non-pathological cases to Hive.

Does this PR introduce any user-facing change?

MSCK repair table with a configured custom partition pattern would previously ignore that pattern and error on finding nonstandard paths. With the code from this PR, it will respect the defined custom pattern when it extracts partition key values from the paths.

Is the change a dependency upgrade?

No

How was this patch tested?

A test was added to TestHiveMetastoreChecker to test the common kinds of custom patterns supported.

[zhangbutao]

@VenkatSNarayanan Please check the failed tests. https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-5591/2/tests. Thx

[VenkatSNarayanan]

@VenkatSNarayanan Please check the failed tests. https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-5591/2/tests. Thx

I can't seem to access that page, do I need to do something to see it?

[zhangbutao]

@VenkatSNarayanan you can login in this page by your github account.

[sonarqubecloud[bot]]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[deniskuzZ]

few tests are failing on Jenkins

org.apache.hadoop.hive.metastore.api.MetastoreException: org.apache.hadoop.hive.metastore.api.MetastoreException: MSCK is missing partition columns under pfile:/home/jenkins/agent/workspace/hive-precommit_PR-5591/ql/target/warehouse/testhivemetastorechecker_db.db/testhivemetastorechecker_table/partdate=2008-01-02/partcity=stockholm
	at org.apache.hadoop.hive.metastore.HiveMetaStoreChecker.checkPartitionDirs(HiveMetaStoreChecker.java:726)

[sonarqubecloud[bot]]

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[VenkatSNarayanan]

@deniskuzZ Fixed the MSCK test failures. There is an Iceberg test failure but it seems unrelated to the changes I've made (it complains about some output being too long).

[deniskuzZ]

re-triggered the build

[github-actions[bot]]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

[deniskuzZ]

hi @VenkatSNarayanan, please address the review comments and re-open the PR if you are still interested to get this in. Thanks!