KAFKA-16285: Make group metadata available when a new assignment is set

[cadonna]

Currently, in the async Kafka consumer updates to the group metadata that are received by the heartbeat are propagated to the application thread in form of an event. Group metadata is updated when a new assignment is received. The new assignment is directly set in the subscription without sending an update event from the background thread to the application thread. That means that there might be a delay between the application thread being aware of the update to the assignment and the application thread being aware of the update to the group metadata. This delay can cause stale group metadata returned by the application thread that then causes issues when data of the new assignment is committed. A concrete example is

producer.sendOffsetsToTransaction(offsetsToCommit, groupMetadata)

The offsets to commit might already stem from the new assignment but the group metadata might relate to the previous assignment.

Summary of testing strategy (including rationale) for the feature or bug fix. Unit and/or integration tests are expected for any behaviour change and system tests should be considered for larger changes.

Committer Checklist (excluded from commit message)

• [ ] Verify design and implementation
• [ ] Verify test coverage and CI build status
• [ ] Verify documentation (including upgrade notes)