bitsail icon indicating copy to clipboard operation
bitsail copied to clipboard

[BitSail][Connector] Doris support deducing partitions.

Open BlockLiu opened this issue 3 years ago • 4 comments

Is your feature request related to a problem? Please describe

Current Doris connector supports write one or more partitions in one job.

To write N partitions, users have to determine the details of the N partitions, including name, start and end range, which is a burden for users. Details can be seen at com.bytedance.bitsail.connector.doris.option.DorisWriterOptions.PARTITIONS.

In some cases, these partitions do match some patterns, like:

{
   "name": "p2022-11-01",
   "start_range": "2022-11-01",
   "end_range": "2022-11-02",
},
{
   "name": "p2022-11-02",
   "start_range": "2022-11-02",
   "end_range": "2022-11-03",
}

So we may want to just give a template and the connector can recognize the pattern and automatically know what the partitions are.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

BlockLiu avatar Nov 09 '22 04:11 BlockLiu

Hi,I am interested in bitsail and doris, can you assign this issue to me? thx!

beyond-up avatar Dec 08 '22 05:12 beyond-up

Hi @beyond-up, thanks for your contribution to the BitSail community. Would you like to join contributor chat group? Here is the QR code. If it expires, you can leave a message here. Look forward to your coming! ContributorGroup

BlockLiu avatar Dec 29 '22 06:12 BlockLiu

如果上面群聊二维码过期了,也可添加微信:13651994967,拉您入群 @beyond-up

yucongcong654 avatar Jan 13 '23 06:01 yucongcong654

Sorry, I closed by mistake, please assign this issue to me again thx ! I will continue。

beyond-up avatar Feb 21 '23 07:02 beyond-up