dinky icon indicating copy to clipboard operation
dinky copied to clipboard

[Bug] [Module Name] Report an error in submitting a task to Paimon written to the Flink Session cluster via dingy

Open wcc1433 opened this issue 10 months ago • 4 comments

Search before asking

  • [x] I had searched in the issues and found no similar issues.

What happened

flink sql: CREATE CATALOGpaimon` WITH ( 'type' = 'paimon', 'warehouse' = 'oss://flink-cluster-k8s/test_paimon', 'fs.oss.endpoint' = 'oss-cn-hangzhou.aliyuncs.com', 'fs.oss.accessKeyId' = '-----------------------------', 'fs.oss.accessKeySecret' = '---------------------------------------' );

CREATE TABLE t_platform_user ( id BIGINT NOT NULL COMMENT '主键', platform_id BIGINT NOT NULL COMMENT '', account_id BIGINT NOT NULL COMMENT '账号id', register_day DATE NOT NULL COMMENT '注册日期', user_id BIGINT COMMENT '用户id', user_register_time TIMESTAMP(3) COMMENT '用户注册时间', device_eid VARCHAR, android_id VARCHAR, manufacturer VARCHAR, sub_provider_email VARCHAR, sub_provider_name VARCHAR, PRIMARY KEY (id) NOT ENFORCED ) WITH ( 'connector' = 'mysql-cdc', 'hostname' = 'mysql.rds.aliyuncs.com', 'port' = '3306', 'username' = 'xxxxx', 'password' = 'xxxxxxxxxxxxx', 'database-name' = 'platform-sync', 'table-name' = 't_platform_user', 'scan.startup.mode' = 'snapshot' );

insert into paimon.ods_platform.t_platform_user select * from t_platform_user;` 错误日志: 2025-04-02 10:05:59 org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate outputs in order. at org.apache.flink.streaming.api.graph.StreamConfig.getVertexNonChainedOutputs(StreamConfig.java:590) at org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriters(StreamTask.java:1743) at org.apache.flink.streaming.runtime.tasks.StreamTask.createRecordWriterDelegate(StreamTask.java:1727) at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:428) at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:380) at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:353) at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:345) at org.apache.flink.streaming.runtime.tasks.StreamTask.(StreamTask.java:335) at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.(OneInputStreamTask.java:78) at jdk.internal.reflect.GeneratedConstructorAccessor23.newInstance(Unknown Source) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source) at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1636) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:749) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.io.InvalidClassException: org.apache.paimon.flink.sink.FlinkStreamPartitioner; local class incompatible: stream classdesc serialVersionUID = -8777668750264956371, local class serialVersionUID = 1301462699919205830 at java.base/java.io.ObjectStreamClass.initNonProxy(Unknown Source) at java.base/java.io.ObjectInputStream.readNonProxyDesc(Unknown Source) at java.base/java.io.ObjectInputStream.readClassDesc(Unknown Source) at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source) at java.base/java.io.ObjectInputStream$FieldValues.(Unknown Source) at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source) at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source) at java.base/java.io.ObjectInputStream.readObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject(Unknown Source) at java.base/java.util.ArrayList.readObject(Unknown Source) at java.base/jdk.internal.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at java.base/java.io.ObjectStreamClass.invokeReadObject(Unknown Source) at java.base/java.io.ObjectInputStream.readSerialData(Unknown Source) at java.base/java.io.ObjectInputStream.readOrdinaryObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject0(Unknown Source) at java.base/java.io.ObjectInputStream.readObject(Unknown Source) at java.base/java.io.ObjectInputStream.readObject(Unknown Source) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:488) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:472) at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:467) at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:422) at org.apache.flink.streaming.api.graph.StreamConfig.getVertexNonChainedOutputs(StreamConfig.java:586) ... 16 more

What you expected to happen

数据正常写入 paimon

How to reproduce

运行环境:dinky 1.2.2 + flink1.20.1 + jdk17 dinky flink jar 截图:

Image

flink session jar 环境截图:

Image

Anything else

No response

Version

1.2.2

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

wcc1433 avatar Apr 02 '25 02:04 wcc1433

Hello @wcc1433, this issue is about CDC/CDCSOURCE, so I assign it to @aiwenmo. If you have any questions, you can comment and reply.

你好 @wcc1433, 这个 issue 是关于 CDC/CDCSOURCE 的,所以我把它分配给了 @aiwenmo。如有任何问题,可以评论回复。

github-actions[bot] avatar Apr 02 '25 02:04 github-actions[bot]

Use add jar.

aiwenmo avatar Apr 03 '25 12:04 aiwenmo

I don't quite understand? What should I do?

wcc1433 avatar Apr 07 '25 01:04 wcc1433

Hello @, this issue has not been active for more than 30 days. This issue will be closed in 7 days if there is no response. If you have any questions, you can comment and reply.

你好 @, 这个 issue 30 天内没有活跃,7 天后将关闭,如需回复,可以评论回复。

github-actions[bot] avatar Jun 01 '25 00:06 github-actions[bot]