Kafka reporter

[jeqo]

Fixes #158

Adds a KafkaReporter to support: native Kafka batching, partitioning by traceId.

[codefromthecrypt]

main thing is handler would be limited to users of brave. otoh brave model has local root etc. both can coexist just depends on what trying to solve. I wouldnt have dual Kafka layers though.

just giving options for direct implementation

On Thu, Aug 29, 2019, 8:58 PM Jorge Quilcate Otoya [email protected] wrote:

@jeqo commented on this pull request.

In kafka/src/main/java/zipkin2/reporter/kafka/KafkaReporter.java https://github.com/openzipkin/zipkin-reporter-java/pull/159#discussion_r319055221 :

• */

• volatile KafkaProducer<byte[], byte[]> producer;
• volatile boolean closeCalled;
• volatile AdminClient adminClient;
• @Override public void report(Span span) {
• if (closeCalled) throw new IllegalStateException("closed");
• metrics.incrementSpans(1);
• int spanSizeInBytes = encoder.sizeInBytes(span);
• metrics.incrementSpanBytes(spanSizeInBytes);
• ProducerRecord<byte[], byte[]> record =

•  new ProducerRecord<>(
  

•    topic,
  

•    span.traceId().getBytes(),
  

•    encoder.encodeList(Collections.singletonList(span)));
  

• getProducer().send(record, (metadata, exception) -> {

I also think a kafka "FinishedSpanHandler" in brave could be more effective. you could bundle messages by local root.

This is something I miss from AsyncReporter where I can group per traceId, and send them together.

local root would solve the same but from brave, right?

kafka "FinishedSpanHandler" in brave

Haven't worked with this API yet tho, but a kafka handler would replace reporter under site-specific config plus grouping spans per local root, right?

I'm just a bit confused on the overlapping between handlers and reporter, and if it's ok to solve this on both sides or pick one.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/openzipkin/zipkin-reporter-java/pull/159?email_source=notifications&email_token=AAAPVVZLO4DG5F56X3RUSB3QG7BXBA5CNFSM4IOOL6MKYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCDDGOBY#discussion_r319055221, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAPVV6NBVB5TKIME3E6UN3QG7BXBANCNFSM4IOOL6MA .

[codefromthecrypt]

" I wouldnt have dual Kafka layers though" means I wouldnt recommend user having to configure a handler and a reporter in a tightly coupled fashion. we can either find a way to pass local root here (ex an extended interface which has a traceid and local root param)

ex partitionedreporter.report(traceId, localRootId, span)

or solve somehow else

[codefromthecrypt]

@jeqo this is going on 5 years old ;) go ahead and close this unless you feel like updating it to SpanHandler in brave. We can also do reporter (mostly for otel)