hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-7146][WIP] Implement secondary index

Open codope opened this issue 1 year ago • 1 comments

Change Logs

This PR is stacked on #11077. Main changes done here:

  • New index type added in MetadataPartitionType
  • Initialization of the new index in HoodieBackedTableMetadataWriter
  • Util methods to support index creation and update in HoodieTableMetadataUtil

Another PR will follow this PR, wherein we will have the query-side integration and more e2e tests.

Impact

Builds secondary index

Risk level (write none, low medium or high below)

low

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

codope avatar Apr 30 '24 17:04 codope

CI report:

  • f39d566fd99b405c4c7b34f2e1c2d5056b9a6d27 Azure: FAILURE
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar May 03 '24 18:05 hudi-bot

Closing it in favor of #11146 as it's becoming hard to maintain two sepate PRs for insert and update.

codope avatar May 06 '24 15:05 codope