Sink-managed consumer group expires after 7 days of no activity in the topic

[fenil25]

Apache Iceberg version

1.7.1 (latest release)

Query engine

Kafka Connect

Please describe the bug 🐞

For the Kafka Connect Iceberg connector, sink-managed consumer group (cg-control-<connector_name>) expires after 7 days of no activity (assuming broker config of offsets.retention.minutes is set to default value of 10080)

When this happens, whats the behavior with the kafka connector? I wasn't able to find the code where the consumer group gets recreated. Do we have to restart the connector for the connector to consume records correctly?

Is there any way to increase the retention by keeping the consumer group alive without changing the broker config?

Willingness to contribute

• [ ] I can contribute a fix for this bug independently
• [X] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• [ ] I cannot contribute a fix for this bug at this time

[github-actions[bot]]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions[bot]]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'