kafka icon indicating copy to clipboard operation
kafka copied to clipboard

KAFKA-17686: AsyncKafkaConsumer.offsetsForTimes() fails with NullPointerException

Open kirktrue opened this issue 1 year ago • 4 comments

The code to convert the Map was initially expressed using the Streams API with collect(Collectors.toMap()). Unfortunately, the implementation of Collector returned by Collectors.toMap() does not support null entries. The KafkaConsumer.offsetsForTimes() API explicitly states that nulls can be present in the returned Map, hence this change.

Committer Checklist (excluded from commit message)

  • [ ] Verify design and implementation
  • [ ] Verify test coverage and CI build status
  • [ ] Verify documentation (including upgrade notes)

kirktrue avatar Oct 03 '24 00:10 kirktrue

KAFKA-14560 will remove the old client protocol API, so do kafka-client 4.0 need to handle null timestamp?

chia7712 avatar Oct 03 '24 06:10 chia7712

KAFKA-14560 will remove the old client protocol API, so do kafka-client 4.0 need to handle null timestamp?

I want to make very sure I understand the KIP:

  • Support for ListOffsets v0 is being removed in 4.0
  • Timestamp (and Offset) are both required in v1 and later
  • The 4.0.0 client should never get to the point where it receives a value of -1 for either the timestamp or offset

Is the above correct?

kirktrue avatar Oct 03 '24 23:10 kirktrue

Separate from this PR, it sounds like the client should remove code that supports those outdated RPC versions.

For example, OffsetAndTimestampInternal appears to exist simply “to allow negative timestamps and offset” values. ListOffsetData is another internal class that is used to compensate for missing values in v0 ListOffset RPC responses. Those classes (and surrounding code) should be removed within the AK 4.0.0 timeframe, right?

kirktrue avatar Oct 04 '24 00:10 kirktrue

Is the above correct?

Great! Glad we're on the same page. :smile:

Those classes (and surrounding code) should be removed within the AK 4.0.0 timeframe, right?

yes, consumer should throw unsupported version if the target broker supports only 0

chia7712 avatar Oct 04 '24 01:10 chia7712

@chia7712 / @lianetm—tests are passing and comments have been addressed. Can you provide another pass of reviews? Thanks!

kirktrue avatar Oct 22 '24 18:10 kirktrue