[HUDI-7855] Add ability to configure bulk insert write parallelism

[rmahindra123]

Change Logs

Integrate StreamContext with StreamSync to get the ability to configure the BULK_INSERT write parallelism based on workload etc. Integrate for both row writer and non-row writer flows.

Impact

Ability to configure the BULK_INSERT write parallelism based on workload etc. Integrate for both row writer and non-row writer flows.

Risk level (write none, low medium or high below)

Low

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change. If not, put "none".

• The config description must be updated if new configs are added or the default value of the configs are changed
• Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the ticket number here and follow the instruction to make changes to the website.

Contributor's checklist

• [ ] Read through contributor's guide
• [ ] Change Logs and Impact were stated clearly
• [ ] Adequate tests were added if applicable
• [ ] CI passed

[hudi-bot]

CI report:

• b9ddc1666ba6ef05c8da1eefa468428db324112f Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build