broadcaster icon indicating copy to clipboard operation
broadcaster copied to clipboard

Fix/22

Open pwoolvett opened this issue 3 years ago • 6 comments

  • test: add concurrent checks

    • group together fixtures for backend testing
    • Add per-backend setup to allow slow initialization. This is specially important as kafka subscribe can return while the consumer is not ready. See related test setup upstream:
      • https://github.com/aio-libs/aiokafka/blob/2c54e10c57760f779961a8c2f5df8ad609ef6983/tests/test_consumer.py#L433
      • https://github.com/aio-libs/aiokafka/blob/2c54e10c57760f779961a8c2f5df8ad609ef6983/tests/_testutil.py#L376
      • https://github.com/aio-libs/aiokafka/blob/2c54e10c57760f779961a8c2f5df8ad609ef6983/tests/_testutil.py#L364
  • chore: update remnants, packages

  • fix(backend/kafka): consumer unsubscribe not awaitable

  • fix(backend/postgres): allow concurrent pubs

    This fix adds a lock (asyncio.Event based) to avoid asyncpg.exceptions._base.InterfaceError: cannot perform operation: another operation is in progress

fixes #22 fixes #42

pwoolvett avatar May 06 '22 19:05 pwoolvett

recently, my project facing some issue

File "*****/python3.9/site-packages/broadcaster/_base.py", line 62, in connect
    await self._backend.connect()
AttributeError: 'Broadcast' object has no attribute '_backend'

seems like this PR doesn't resolve slow backend initialization issue

skyforeverRivian avatar May 28 '24 20:05 skyforeverRivian

Why didn't this merge? If I fixed its conflicts, would it be accepted?

logankaser avatar Jun 11 '24 23:06 logankaser

@logankaser let's have it split into multiple PRs. I believe there is only one issue related to postgres left.

alex-oleshkevich avatar Jun 12 '24 08:06 alex-oleshkevich

I think there are two:

  1. This issue, the connection is not guarded by a lock, so it can be accessed while the connection is active. Can be with using a asyncio lock like this PR did, or better something like https://github.com/permitio/broadcaster/blob/master/broadcaster/_backends/postgres.py
  2. #66 Postgres backend cannot gracefully handle connection loss

logankaser avatar Jun 12 '24 17:06 logankaser

https://github.com/encode/broadcaster/pull/66 looks like a good merge candidate, but I did not test it yet.

alex-oleshkevich avatar Jun 12 '24 17:06 alex-oleshkevich

Any update on this issue? I'm particularly interested in #22 fix 🙏 .

Trinkes avatar Aug 17 '24 16:08 Trinkes