Leonard Xu comments

Results 145 comments of


                                            Leonard Xu

http://spark-notebook.io is down?

It seems [http://spark-notebook.io](http://spark-notebook.io) is still down today. Have any official can help fix the website?

Asked to waitFor but it was never even start()'ed - since 2.4.0

Same exception for me, run normally in my local mac env, failed in linux server env. Does the this issue have any progress ?

[tikv][cdcclient] The NPE error occurs in CDCClient

@zz-jason I'd like to fix it this week.

[common] Add Flink 1.15 support.

> when support cdc to elasticsearch 8.x for flink cdc ? You can open a issue firstly @CaoYunzhou

[Postgres] Use Incremental Snapshot Framework for Postgres CDC Connector

@qiao-x Thanks for taking this ticket, assigned to you.

fix startupOptions mode bug

@TeriMoni Thanks for your contribution. But we need to improve the PR to really support these options, currently the underlying code does not support these options, thus your change is...

以后CDC里CREATE、DROP、ALTER TABLE这种想要如何实现？

> 另外，其他一些信息也可以考虑进去，比如Heartbeat：现在如果把CDC的维表用作Versioned Table直接用来join，会因为维表数据变更慢，导致维表这边的Watermark涨不上去，其实用Heartbeat消息来推动Watermark上涨，才是最合理的办法。想法不错，Heartbeat消息确实是个很有效的输入

以后CDC里CREATE、DROP、ALTER TABLE这种想要如何实现？

> > watermark 可以 pushdown 到 cdc source 里面，这样 heartbeat 数据不用让 flink 框架感知。 > > 其实在我看来，changelog中有必要包含heartbeat信息，因为这样才能知道真正的watermark位置，下游系统其实需要这个信息，比如把changelog输出到Kafka、Iceberg之后，他们可以去记录真实的watermark +1 changelog中目前只有数据信息，但我理解 @wuchong 的意思是 heartbeat 信息可以通过pushdown到cdc source 里作为watermark，这样heartbeat信息就可以走watermark流，一路下传给kafka，iceberg，hudi，确实这些下游系统时需要这部分信息的。

问题：生产环境下connector可能会对mysql server的造成i/o压力? 是否支持mysql server的主从切换？

filter & project pushdown 只是后续的优化方向，还未开始做...

[oracle]oracle cdc supports synchronizing oracle slave nodes

Thanks @waywtdcc for opening this PR, but could you open an issue to describe the feature before open a PR？