flink-cdc initial mode don't get data when new create db and table and restart job

Describe the bug(Please use English) A clear and concise description of what the bug is.

Environment :

Flink version : 1.13.5
Flink CDC version: 2.2.0
Database and version: mysql 5.7.38

To Reproduce Steps to reproduce the behavior:

Thes test data :

The test code : MySqlSource<String> mysqlSource = MySqlSource.<String>builder() .hostname("192.168.XX.XX") .port(3306) .username("root") .password("root")

         .databaseList("db01")
         .tablList(".*")
         .startupOptions(StartupOptions.initial())
         .deserializer(new JsonDebeziumDeserializationSchema())           
         .build();

The error : 10 mins later error comes: 2022-07-10 10:54:17.402 [main] INFO java.lang.Class - 4.开始执行同步任务... 2022-07-10 11:05:23.372 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:05:23.372 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:06:24.407 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:06:24.407 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:07:25.420 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:07:25.420 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:08:26.430 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:08:26.430 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:09:27.460 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121] 2022-07-10 11:09:27.460 [SourceCoordinator-Source: MySQL-Cdc-Source] ERROR org.apache.flink.runtime.source.coordinator.SourceCoordinator - Uncaught exception in the SplitEnumerator for Source Source: MySQL-Cdc-Source while handling operator event RequestSplitEvent (host='') from subtask 0. Triggering job failover. org.apache.flink.util.FlinkRuntimeException: InterruptedException while waiting for asynchronously snapshot split at com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.getNext(MySqlSnapshotSplitAssigner.java:235) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.assigners.MySqlHybridSplitAssigner.getNext(MySqlHybridSplitAssigner.java:125) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.assignSplits(MySqlSourceEnumerator.java:193) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at com.ververica.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator.handleSplitRequest(MySqlSourceEnumerator.java:109) ~[flink-connector-mysql-cdc-2.2.0.jar:2.2.0] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$handleEventFromOperator$1(SourceCoordinator.java:172) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$8(SourceCoordinator.java:331) ~[flink-runtime_2.12-1.13.5.jar:1.13.5] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_121] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_121] at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]

Jul 10 '22 03:07 deepthinkin

later, i use the way of remote debug found the error that dbs contains all table no primary key. but it is not friendly, why not throws errors.

Jul 11 '22 00:07 deepthinkin

@deepthinkin thank you for save my life.

the consume table should has primary key for buildSplitQuery . the error message is not friendly.

Jan 11 '23 04:01 geosmart

@deepthinkin thank you for save my life.

the consume table should has primary key for buildSplitQuery . the error message is not friendly.

Could you please clarify exactly what buildSplitQuery is and where you configure it?

Sep 18 '23 21:09 EGitDeveloper

@deepthinkin thank you for save my life. the consume table should has primary key for buildSplitQuery . the error message is not friendly.

Could you please clarify exactly what buildSplitQuery is and where you configure it?

first the consume table should has primary key. buildSplitQuery is the internal function of cdc :

Oct 24 '23 02:10 geosmart

Closing this issue because it was created before version 2.3.0 (2022-11-10). Please try the latest version of Flink CDC to see if the issue has been resolved. If the issue is still valid, kindly report it on Apache Jira under project Flink with component tag Flink CDC. Thank you!

Feb 28 '24 15:02 PatrickRen