dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Improvement][datax] Support DataX parameter `batchSize` for writer

Open bromine0x23 opened this issue 1 year ago • 5 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Description

Currently DataX task does not support writer parameter batchSize, therefore only the default values can be used, which is 2048.

However this default value is too small for some databases, such as the ClickHouse I'm using. The example value in DataX is 65536.

So I need the support of batchSize parameter.

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

bromine0x23 avatar Oct 31 '24 03:10 bromine0x23

i think can try do it

YOMO-Lee avatar Nov 02 '24 02:11 YOMO-Lee

Using a custom template

xinxingi avatar Nov 06 '24 02:11 xinxingi

Hi, I’d like to take this issue and submit a PR. Could you please assign it to me? I will try to do better this time.

Oscarcheng0312 avatar Mar 31 '25 08:03 Oscarcheng0312

@bromine0x23 @SbloodyS can you assign me this i will do this soon

namanONcode avatar Dec 04 '25 16:12 namanONcode

这是来自QQ邮箱的自动回复邮件。   您好,我最近正在休假中,无法亲自回复您的邮件。我将在假期结束后,尽快给您回复。

YOMO-Lee avatar Dec 04 '25 16:12 YOMO-Lee