ydb-java-sdk icon indicating copy to clipboard operation
ydb-java-sdk copied to clipboard

Batch all messages in topic reader, Refactor internal event queue

Open pnv1 opened this issue 1 year ago • 1 comments

Both async and sync readers transfer messages in batches that are received from server as is. Sometimes it hurts performance in user scenarios where on each received batch user perform a long running operation. More batches leads to more long-running operations. The idea is to batch all batches received from server into one. Therefore, user will receive all available messages in one batch.

pnv1 avatar Aug 02 '24 11:08 pnv1

Also, system events such as Start/Stop PartitionSession for every partition should be put into the same message queue to guarantee the order of callback calls not only between the data events but between data and system events too.

pnv1 avatar Feb 25 '25 06:02 pnv1

Implemented here https://github.com/ydb-platform/ydb-java-sdk/pull/530/

alex268 avatar Sep 04 '25 11:09 alex268