kafka KAFKA-10199: Integrate Topology Pause/Resume with StateUpdater

When a topology is paused / resumed, we also need to pause / resume its corresponding tasks inside state updater.

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Sep 19 '22 01:09 guozhangwang

cc @cadonna please let me know if you like the current approach i.e. exposing updating / paused tasks from state updater, which may be stale with newly paused / resumed tasks (and they will be handled in the next iteration then). If it lgty I will add unit tests.

Sep 19 '22 01:09 guozhangwang

@cadonna ready for another quick look.

Sep 21 '22 20:09 guozhangwang

After discussed with @cadonna offline, we decide to first try with the simpler approach which let the state updater thread periodically check the source-of-truth topology metadata directly and pause / resume / checkpoint tasks. If pause / resume become a popular feature and the asynchronous procedure's delay becomes confusing to users, we would switch to a smaller-latency event-driven mechanism.

Sep 27 '22 18:09 guozhangwang