streamnative
[FEATURE] Can kop support setting the entry format of topic level
Is your feature request related to a problem? Please describe. Currently kop only supports entryFormat at the agent level. Once kop is configured with entryFormat, it will no longer be variable, and all topics will execute this entryFormat. If we want a topic to support pulsar consumption, or we want to reduce the conversion between pulsar messages and kafka messages for high-traffic topics, and hope to change the topic entryFormat to kafka, we cannot do this now except for restarting the broker.
Describe the solution you'd like kop supports dynamically changing the topic-level entryFormat according to everyone’s needs.
If we want a topic to support pulsar consumption
I'm working on supporting Pulsar consumption for messages whose entry format is Kafka recently. After testing the demo in my local env, I'll submit a PIP on Pulsar side.
If we want a topic to support pulsar consumption
I'm working on supporting Pulsar consumption for messages whose entry format is Kafka recently. After testing the demo in my local env, I'll submit a PIP on Pulsar side.
Thank you for your reply. I also thought of this. Supporting the consumption of Kafka messages in pulsar, because this requires code changes in pulsar, so I gave this issue to kop first. If pulsar supports the consumption of Kafka messages, this would be a great thing. :)
Yeah, you can see the first optional task in https://github.com/streamnative/kop/issues/581. Since we tracked the task in StreamNative's private repo, this issue was not updated recently.
Yeah, you can see the first optional task in #581. Since we tracked the task in StreamNative's private repo, this issue was not updated recently.
Good task. Let us wait for pulsar to support the consumption of kafka messages.
