Guidance on standalone consumer implementation

[pdonorio]

Hi, we are trying to implement what is suggested in https://medium.com/@ravisharma911993/kafka-broadcasting-messages-without-consumer-groups-5a374fcfb7bc - a standalone consumer.

The use case: we have dynamic pods coming alive consuming from latest, die and not set any offset.

This would require, among other things, to set the group id to empty/none. Based on the post this is ok in Java, but in python we get

Failed to subscribe to topics: KafkaError{code=_UNKNOWN_GROUP,val=-179,str="Failed to set subscription: Local: Unknown group"}

Are there any plan to support this use case? What is the current reason we do not allow for empty group?

Thank you 🙏

[pranavrth]

We support this usecase in librdkafka. We should be supporting in python client as well but there was an issue earlier and fix for it was to keep group.id mandatory. I don't know the context of this decision.

I will discuss this within the team.

[pdonorio]

We support this usecase in librdkafka. We should be supporting in python client as well but there was an issue earlier and fix for it was to keep group.id mandatory. I don't know the context of this decision.

I will discuss this within the team.

thank you @pranavrth - this is really helpful info.

I'll wait for more guidance 🙏