azure-sdk-for-python icon indicating copy to clipboard operation
azure-sdk-for-python copied to clipboard

[EventHubs] update sync receive client ready flow

Open swathipil opened this issue 3 years ago • 3 comments

When reading from 32 partitions, the sync consumers were taking a long time to connect, b/c the client ready check was happening in a while loop. Each consumer was blocking while checking that the client was ready. Overall, this resulted in very slow connection times for the consumer client.

This is updating the consumers so that the client ready check is no longer blocking.

swathipil avatar Nov 09 '22 22:11 swathipil

/azp run python - eventhub - tests

swathipil avatar Nov 09 '22 22:11 swathipil

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Nov 09 '22 22:11 azure-pipelines[bot]

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-eventhub

azure-sdk avatar Nov 09 '22 23:11 azure-sdk

Awesome thanks for this! Is there any investigation needed for async? Or the producers? (If so, we could probably open issues to address as needed). The live tests were all good?

Going to run the live tests on this to double check that this doesn't result in any issues.

Producers and async consumer don't need to be updated. + The perf connect issue goes away with just the sync consumer fix.

More details: Async consumer open was always calling client ready in a loop (for uamqp as well). As you mentioned, I think this is because it calls client_ready() and asyncio.sleep with await, so it wouldn't be blocking.

Sync and async producers were also always calling client_ready() in a loop.

swathipil avatar Nov 14 '22 18:11 swathipil

/azp run python - eventhub - tests

swathipil avatar Nov 14 '22 18:11 swathipil

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Nov 14 '22 18:11 azure-pipelines[bot]

/azp run python - eventhub - tests

swathipil avatar Nov 16 '22 19:11 swathipil

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines[bot] avatar Nov 16 '22 19:11 azure-pipelines[bot]