open-telemetry
OTEL Kafka Exporter
Is your feature request related to a problem?
In some cases it is good to export spans, logs, and metrics direct to Kafka, rather than elsewhere. For example, all your applications could publish to a Kafka cluster/s, with an OpenTelemetry Collector configured to export elsewhere. If you then wanted to change where you wanted to export your telemetry data you would only need to update the configuration for a single deployment
Describe the solution you'd like
Exporters that publish to Kafka. Defaults should match those of the Kafka Receiver from the opentelemetry-collector. For example the default topic for traces would be otlp_spans
Describe alternatives you've considered
No response
Additional Context
No response
Would you like to implement a fix?
Yes
Tip
React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.
@jackburridge What about having an OTel collector on the side re-exporting to kafka with the kafka exporter?
@xrmx agreed you could, but this would add additional compute (not a huge amount). The implementation of a Kafka exporter should be relatively simple, most of the protobuf encoding code is in opentelemetry-exporter-otlp-proto-common already.
If you were to use it in something like AWS Lambda for example you could use https://github.com/open-telemetry/opentelemetry-lambda but this adds a collector to the Lambda, why increase the size if you don't have to