bitsail icon indicating copy to clipboard operation
bitsail copied to clipboard
verified publisher icon bytedance

[Improve][Connector] Add split progress in jdbc/mongo

Open lichang-bd opened this issue 3 years ago • 6 comments

Description

In the Batch scenario, task progress can be measured according to Complete Split/ Total Split, Especially in JDBC and Mongo scenarios, we can add split processing information

1000 can be changed to parameter configuration image

JDBC lack of split progress information in log image

image

BitSail Component or Code Module

BitSail Connector

Are you willing to submit PR?

  • [ ] Yes, I am willing to submit a PR!

Code of Conduct

lichang-bd avatar Feb 13 '23 07:02 lichang-bd

please assign to me, thanks

CodingGPT avatar Feb 14 '23 15:02 CodingGPT

Hi CodingGPT, thanks for your contribution to the BitSail community. we build a group for all contributor.Here is the QR code. If it expires, you can add WeChat ID 13651994967. Look forward to your arrival! @CodingGPT 1321676443482_ pic

yucongcong654 avatar Feb 15 '23 06:02 yucongcong654

how about this issue, any problem about this, we can disscuss @CodingGPT

lichang-bd avatar Feb 27 '23 03:02 lichang-bd

Sorry I accidentally added the comment above and I can not delete it , please delete it for me, thx! @lichang-bd

beyond-up avatar Mar 03 '23 08:03 beyond-up

I am testing, found range and currentTaskId is null, the log like this com.bytedance.bitsail.connector.legacy.mongodb.source.MongoDBInputFormat [] - Start to process range info: [null,null] [0/1]

How to debug this program in idea on my local machine?

CodingGPT avatar Mar 05 '23 15:03 CodingGPT