flink icon indicating copy to clipboard operation
flink copied to clipboard
verified publisher icon apache

[FLINK-25055][runtime] Support listen and notify mechanism for partition request

Open FangYongs opened this issue 4 years ago • 2 comments

What is the purpose of the change

Support listen and notify mechanism for partition request

Brief change log

  • Add partition request notifier to ResultPartitionManager when given partition is not registered yet.
  • Notify the given partition request notifier when the given partition is registered to the ResultPartitionManager.
  • Remove the timeout partition request notifier from ResultPartitionManager and send PartitonNotFoundException the the netty client.

Verifying this change

This change added tests and can be verified as follows:

  • Add create partition request notifier in test ResultPartitionManagerTest
  • More test cases will be added later

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no) no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) no
  • The serializers: (yes / no / don't know) no
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know) no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) no
  • The S3 file system connector: (yes / no / don't know) no

Documentation

  • Does this pull request introduce a new feature? (yes / no) no
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

FangYongs avatar Apr 06 '22 14:04 FangYongs

CI report:

  • 50dd2d03c1762fe0710b636061350f017bf0b651 Azure: FAILURE
Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

flinkbot avatar Apr 06 '22 14:04 flinkbot

Hi @KarmaGYZ @pnowojski Thanks for your reviews and I'm very sorry for updating this PR so late. I have rebased the code from master and updated the codes in the PR via your comments. I will focus on this PR and hope to complete it, can you help to continue reviewing it when you're free? Thanks and sorry again :)

FangYongs avatar Sep 19 '22 10:09 FangYongs

JFYI, I've refactor this PR, addressed all the exisiting comments and rebase on the lastest master in #23565

KarmaGYZ avatar Oct 23 '23 07:10 KarmaGYZ