hudi icon indicating copy to clipboard operation
hudi copied to clipboard

[HUDI-7484] Enable url encoded partitioning if hive style partition enabled

Open codope opened this issue 1 year ago • 1 comments

Change Logs

If hive style partitioning is enabled, then url encoding is also enabled. We need to do so otherwise the partition structure is awkward in some cases. For example, a partition field with values like "dd/mm/yy" will be written as three-level directory where the first level is partition=dd and then its child is mm, and then yy.

partition=dd
                     |- mm
                           |- yy

With URL encoding enabled, it will be partition=dd%2Fmm%2Fyy.

Impact

Correct partition strucutre. However, for a small subset of users who had partitions in above format and hiev style paritioning enabled, this is going to be a physical layout change.

Risk level (write none, low medium or high below)

medium

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.

Contributor's checklist

  • [ ] Read through contributor's guide
  • [ ] Change Logs and Impact were stated clearly
  • [ ] Adequate tests were added if applicable
  • [ ] CI passed

codope avatar Sep 26 '24 17:09 codope

CI report:

  • ff35184cef88e34aabb33e75ce724a2dd1812b45 Azure: SUCCESS
Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

hudi-bot avatar Sep 30 '24 04:09 hudi-bot