streamnative
[BUG] Consumers with read_committed still see some messages from aborted transactions
Describe the bug If a transaction is aborted/rolled back, I think its messages should not be visible to consumers with setting read_committed. However, if there is some processing delay before abortTransaction() is called (75ms or more) then the consume will sometimes receive the messages.
To Reproduce Steps to reproduce the behavior:
- Start a transactional producer and a read_committed consumer
- The producer begins a transaction, sends a message, pauses for 100ms, then calls abortTransaction().
- Repeat step 2 several times.
Expected behavior Expect the consumer to receive NONE of the messages.
Screenshots If applicable, add screenshots to help explain your problem.
Additional context pulsar-cloud:2.9.2.16 with KoP pulsar-protocol-handler-kafka-2.9.2.16.nar Running on my local machine against the kj-kop cluster in sndev (sncloud-production)
Source code and config
If the transactionProcessingTimeMs in the config is 75ms, my consumer receives about 10-15% of the messages from aborted transactions. If the transactionProcessingTimeMs is 250ms, my consumer receives 40-50% of the messages.
