weiboad
How to set consumerId
Hello,
I have explored this module and it's difficult for me to set consumerId, I have seen there is an option to set clientid, but I guess clientid is different consumerid is different.
Looking for help
@khizarsonu I think your consumerId is the memberID generated by join group, not allowed to set up
You can refer to
http://kafka.apache.org/protocol.html#The_Messages_JoinGroup
isn't there any way to manually & explicitly set consumer id while creating a single consumer? Kafka's API in other languages do provide such options.
@khizarsonu can you please describe the reason why you need to set the consumer.id? Aren't you confusing it with group.id?
isn't there any way to manually & explicitly set consumer id while creating a single consumer? Kafka's API in other languages do provide such options.
Not in the new (well, not so new) consumer configuration: http://kafka.apache.org/documentation.html#newconsumerconfigs
There was an option in the past but it was removed because it didn't make sense to have it available.
Expanding what @nmred said: the consumer.id is an option used by the group manager to track which consumers are in the ACTIVE state within a group, it must be generated automatically to avoid clashes and potentially mess up with that verification.
Actually, I was using single consumer, NOT consumer group. And in my use case, I need to change my machine every few weeks. But when my machine is changed, somehow my consumer ID is changed too and so I start receiving messages from earliest available offset since Kafka considers me to be a different consumer which I'm not. So I was looking for a way to somehow set the consumer.id explicitly so that even if my machines gets changed Kafka should remember me as old consumer and should give me messages after last committed offset and not from earliest available offset
@ashishnetworks in Kafka every consumer is part of a group, like always. You can have a single consumer in a group but it's the group who is responsible for controlling the offset of the consumers related to it. So what you need is to indeed set the group.id and that will do what you want.