[OTLP] Generate Sentry issues based on OTel span exception events

[gregoiredx]

Problem Statement

Currently, the OpenTelemetry SentrySpanProcessor does not interpret span events at all.

This means that when a span has an exception event attached, there's no issue visible in Sentry when using INSTRUMENTER.OTEL and the OTel Python SDK.

When using the OTel API to record and exception using trace.get_current_span().record_exception(error), it has no effect on Sentry issues, while one would expect it to do something equivalent to sentry_sdk.capture_exception(error).

Solution Brainstorm

In the docs, there's already an idea of what could be done, at least in TypeScript, see https://develop.sentry.dev/sdk/telemetry/traces/opentelemetry/#step-7-define-generatesentryerrorsfromotelspan

In OpenTelemetry, spans can have exception events. These have a stacktrace, message, and type. We want to convert these to Sentry errors and attach them to the trace.

function generateSentryErrorsFromOtelSpan(otelSpan) {
  otelSpan.events.forEach(event => {
    // Only convert exception events to Sentry errors.
    if (event.name !=== 'exception') {
      return;
    }
    const attributes = event.attributes;
    const message = attributes[SemanticAttributes.EXCEPTION_MESSAGE];
    const syntheticError = new Error(message);
    syntheticError.stack = attributes[SemanticAttributes.EXCEPTION_STACKTRACE];
    syntheticError.name = attributes[SemanticAttributes.EXCEPTION_TYPE];
    Sentry.captureException(syntheticError, {
      contexts: {
        otel: {
          attributes: otelSpan.attributes,
          resource: otelSpan.resource.attributes,
        },
        trace: {
          trace_id: otelSpan.spanContext().traceId,
          span_id: otelSpan.spanContext().spanId,
          parent_span_id: otelSpan.parentSpanId,
        },
      },
    });
  });
}

But it's not that easy to transpose in Python since sentry_sdk.capture_exception(error) relies a lot on having an actual exception instance which we don't have in SentrySpanProcessor.on_end(otel_span) where we could generate Sentry issues from OTel span events (we do have the exception type, message, and stacktrace, though).

[linear[bot]]

PY-1973

[sl0thentr0py]

@gregoiredx thanks for the issue. We're actually in the process of rolling out first-class OTLP support - https://github.com/getsentry/sentry-python/pull/4877 and we will encourage users to move away from the SpanProcessor once it's released.

As part of that new OTLPIntegration, we could indeed make sure that we intercept errors reported via trace.get_current_span().record_exception(error) and make sure they are trace connected. Will let you know once it's all ready!

[gregoiredx]

@sl0thentr0py thanks for your rapid answer, yes please let me know when OTLPIntegration is released!

On a close subject, I tested the Sentry OTLP endpoint and it looks like similarly it's not taking into account exception events to create issues. It looks to me like it should. WDYT? Should I create an issue elsewhere?

[sl0thentr0py]

by itself it will not, correct. But we will make it very easy to integrate Sentry into an existing otel setup with an extra integration in the Sentry SDK that you just have to turn on.

This issue is sufficient for tracking that feature!

[kerenkhatiwada]

Hey team, any update on this issue? Have another user running onto this here.

[sl0thentr0py]

next week!