beam icon indicating copy to clipboard operation
beam copied to clipboard

KafkaIO should raise an error if both .withReadCommitted() and .commitOffsetsInFinalize() are used

Open damccorm opened this issue 3 years ago • 1 comments

Read committed tells KafkaIO to only read messages that are already committed which means that committing offsets in finalize is a no-op.

Users should be using one or the other and is similar to making sure that either auto commit is enabled in the reader of KafkaIO does the committing which we have error checking for already.

Imported from Jira BEAM-13353. Original Jira may contain additional context. Reported by: lcwik.

damccorm avatar Jun 04 '22 21:06 damccorm

slighty related https://github.com/apache/beam/issues/22631

nbali avatar Aug 09 '22 01:08 nbali