sentry-java icon indicating copy to clipboard operation
sentry-java copied to clipboard
verified publisher icon getsentry

NoClassDefFoundError: io/opentelemetry/semconv/SemanticAttributes after OpenTelemetry upgrade

Open erkieh opened this issue 9 months ago • 4 comments

Integration

sentry-opentelemetry-core

Java Version

21

Version

8.11.1

Steps to Reproduce

Use newest Micronaut framework with open telemetry.

It was removed from open-telemetry in 1.30.0

https://github.com/open-telemetry/semantic-conventions-java/releases/tag/v1.30.0-rc.1

Using sentry with this version causes an error. I cannot downgrade as it is a transitive/dependency.

Expected Result

No Error

Actual Result

java.lang.NoClassDefFoundError: io/opentelemetry/semconv/SemanticAttributes
	at io.sentry.opentelemetry.OpenTelemetryAttributesExtractor.extractUrl(OpenTelemetryAttributesExtractor.java:116)
	at io.sentry.opentelemetry.OtelInternalSpanDetectionUtil.isSentryRequest(OtelInternalSpanDetectionUtil.java:31)
	at io.sentry.opentelemetry.SentrySampler.shouldSample(SentrySampler.java:53)
	at io.opentelemetry.sdk.trace.SdkSpanBuilder.startSpan(SdkSpanBuilder.java:192)
	at io.opentelemetry.instrumentation.api.instrumenter.Instrumenter.doStart(Instrumenter.java:196)
	at io.opentelemetry.instrumentation.api.instrumenter.Instrumenter.start(Instrumenter.java:135)
	at io.opentelemetry.instrumentation.jdbc.internal.OpenTelemetryStatement.wrapCall(OpenTelemetryStatement.java:310)
	at io.opentelemetry.instrumentation.jdbc.internal.OpenTelemetryStatement.wrapCall(OpenTelemetryStatement.java:299)

erkieh avatar May 04 '25 18:05 erkieh

Hey @erkieh thanks for opening this issue. Sentry is currently relying on version 1.28.0-alpha of the semconv dependencies as defined by OpenTelemetry here: https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/v2.10.0/dependencyManagement/build.gradle.kts#L49 since we get this by using version 2.10.0 of the OpenTelemetry Java Agent as a base for our own Java Agent.

Can you please pin the version to 1.28.0-alpha or figure out what's causing your build system to use a newer version of OpenTelemetry semantic conventions?

adinauer avatar May 05 '25 08:05 adinauer

@adinauer I am using Micronaut Tracing https://github.com/micronaut-projects/micronaut-tracing/blob/7.0.x/gradle/libs.versions.toml

erkieh avatar May 05 '25 10:05 erkieh

We can bump the OpenTelemetry version in Sentry but I can't give an ETA at this point.

adinauer avatar May 06 '25 08:05 adinauer

Thank you.

erkieh avatar May 06 '25 08:05 erkieh

Please consider using at least a released version of Semconv (currently available: 1.34.0) instead of 1.28.0-alpha.

This is an issue for us while trying to upgrade to SpringBoot 3.5.3+.

jrs666666 avatar Jun 25 '25 10:06 jrs666666

Thanks for reaching out, we'll bump prio.

adinauer avatar Jun 25 '25 14:06 adinauer

@erkieh @jrs666666 we have just released https://github.com/getsentry/sentry-java/releases/tag/8.17.0 which bumps the OpenTelemetry version including instrumentation and semconv. Can you please check if this resolves the issue?

adinauer avatar Jul 10 '25 06:07 adinauer

@adinauer Well, our application is kind of "working" with SpringBoot 3.5.3 and 8.0.17, at least it doesn't crash while logging or tracing.

But we struggle to get a predictable behaviour of seeing logged error and exception at Sentry.io every time we recompile/restart the application. Currently we stripped down to "sentry-spring-boot-starter-jakarta:8.17.0" and "sentry-logback:8.17.0" w/o permanent success in getting our data sent to Sentry.io.

jrs666666 avatar Jul 18 '25 10:07 jrs666666

Hi @jrs666666

  1. Does it work consistently with a different (older) version of Sentry SDK?
  2. Have you enabled debug to check the logs for any hints?
  3. How are you setting up Sentry and OpenTelemetry?

adinauer avatar Jul 18 '25 12:07 adinauer

@adinauer

  1. It worked consistently using SpringBoot 3.4.7, SpringCloud 2024.0.2, Sentry SDK 7.22.6. It doesn't work most of the time using SpringBoot 3.5.3, SpringCloud 2025.0.0, Sentry SDK 8.17.0.
  2. I enabled debugging during our local testing. There were no differences inside the debug logs between an application which published events to Sentry.io and an application started without publishing events.
  3. We're using an agentless approach. Our dependencies consist of
+--- io.sentry:sentry-spring-boot-starter-jakarta:8.17.0
|    +--- io.sentry:sentry-spring-boot-jakarta:8.17.0
|    |    +--- io.sentry:sentry:8.17.0
|    |    +--- io.sentry:sentry-spring-jakarta:8.17.0
|    |    |    +--- io.sentry:sentry:8.17.0
|    |    |    \--- io.sentry:sentry-reactor:8.17.0
|    |    |         \--- io.sentry:sentry:8.17.0
|    |    \--- io.sentry:sentry-reactor:8.17.0 (*)
|    \--- org.springframework.boot:spring-boot-starter:3.5.0 -> 3.5.3 (*)
+--- io.sentry:sentry-logback:8.17.0
|    \--- io.sentry:sentry:8.17.0
...
|    +--- io.opentelemetry:opentelemetry-api -> 1.49.0
|    |    \--- io.opentelemetry:opentelemetry-context:1.49.0
...
+--- io.sentry:sentry-spring-boot-starter-jakarta:8.17.0 (n)
+--- io.sentry:sentry-logback:8.17.0 (n)

It seems that the behaviour of whether getting events inside Sentry.io or not is highly randomized. It changes per build, even if just some unimportant empty line is added to the logback-spring.xml. According to our dependency tree, there's no mixup of different version of the same library, which probably could explain such an erratic behaviour.

jrs666666 avatar Jul 18 '25 15:07 jrs666666

Thanks @jrs666666 we'll try to reproduce and report back.

adinauer avatar Jul 21 '25 07:07 adinauer

@jrs666666 can you please share your application.properties / application.yml?

Did you add the sentry-opentelemetry-agentless-spring dependency or sentry-opentelemetry-agentless?

Are you passing any ENV variables?

adinauer avatar Jul 21 '25 08:07 adinauer

@adinauer For our tests we neither use sentry-opentelemetry-agentless-spring nor sentry-opentelemetry-agentless. See provided excerpt of dependencies in https://github.com/getsentry/sentry-java/issues/4368#issuecomment-3089887169. We use io.sentry:sentry-spring-boot-starter-jakarta, because we need to get exceptions and error logs reliably reported before we could possibly think about traces.

  • effective application.yml:
sentry:
  traces-sample-rate: 1.0 # sampling applied by sentry; spring otel already applied sampling
  instrumenter: otel
#  debug: false
#  diagnostic-level: DEBUG
  dsn: https://xxxxxxxxxx.ingest.us.sentry.io/yyyyyyyyyy
  environment: preview
#  logging:
#    enabled: true # default is true
#    minimum-event-level: error

#logging:
#  level:
#    io.sentry: DEBUG
#    io.opentelemetry: DEBUG
#    web: DEBUG

management:
  tracing:
    sampling:
      probability: 1.0 # we need to see issues within PREVIEW environment
  • Environment variables:
      SPRING_PROFILES_ACTIVE:       preview,kubernetes
  • logback-spring.xml:
<configuration>

    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <include resource="org/springframework/boot/logging/logback/console-appender.xml"/>

    <springProfile name="default">
        <root level="INFO">
            <appender-ref ref="CONSOLE"/>
        </root>
    </springProfile>

    <springProfile name="dev, preview, prod">
        <appender name="CONSOLE_ECS" class="ch.qos.logback.core.ConsoleAppender">
            <encoder class="zzzzzzzzzzzzzzzzzzzzzz"/>
        </appender>
        <appender name="Sentry" class="io.sentry.logback.SentryAppender">
            <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
                <level>ERROR</level>
            </filter>
        </appender>

        <root level="INFO">
            <appender-ref ref="CONSOLE_ECS"/>
            <appender-ref ref="Sentry"/>
        </root>
    </springProfile>
</configuration>

jrs666666 avatar Jul 21 '25 10:07 jrs666666

@jrs666666 are you manually configuring OpenTelemetry for use with Sentry? If so, could you please share that code?

adinauer avatar Jul 21 '25 10:07 adinauer

@adinauer Currently, we focus on getting exceptions and error logs reliably reported.

Why do you insist on getting information about tracing? For our tests we removed our specific tracing library. If Sentry isn't able to report exceptions and error logs reliable we won't care about tracing at all.

jrs666666 avatar Jul 21 '25 11:07 jrs666666

@jrs666666 Depending on how you set up Sentry and OpenTelemetry, the SDK behaves differently. This is not just about tracing but also about propagation of Scopes and potential re-init of the SDK that could cause events to be dropped from the queue.

adinauer avatar Jul 21 '25 11:07 adinauer

@adinauer Our tracing library was removed from the test application. Therefore no manual OpenTelemetry configurations are left.

jrs666666 avatar Jul 21 '25 14:07 jrs666666

Thanks for the update. So far I'm unable to reproduce.

There were no differences inside the debug logs

Does that mean you're seeing the missing events in log output?

Here's an example of what to look for:

DEBUG: Serializing object: {"timestamp":"2025-07-21T14:02:14.591Z","message":{"formatted":"Trying person with id=10","message":"Trying person with id={}","params":["10"]},"logger":"io.sentry.samples.spring.boot.jakarta.PersonController","exception":{"values":[{"type":"RuntimeException","value":"error while loading","module":"java.lang","thread_id":60,"stacktrace":{"frames":[{"filename":"ApplicationFilterChain.java","function":"doFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":140,"native":false},{"filename":"ApplicationFilterChain.java","function":"internalDoFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":164,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"RequestContextFilter.java","function":"doFilterInternal","module":"org.springframework.web.filter.RequestContextFilter","lineno":100,"native":false},{"filename":"ApplicationFilterChain.java","function":"doFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":140,"native":false},{"filename":"ApplicationFilterChain.java","function":"internalDoFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":164,"native":false},{"filename":"DelegatingFilterProxy.java","function":"doFilter","module":"org.springframework.web.filter.DelegatingFilterProxy","lineno":278,"native":false},{"filename":"DelegatingFilterProxy.java","function":"invokeDelegate","module":"org.springframework.web.filter.DelegatingFilterProxy","lineno":362,"native":false},{"filename":"WebMvcSecurityConfiguration.java","function":"doFilter","module":"org.springframework.security.config.annotation.web.configuration.WebMvcSecurityConfiguration$CompositeFilterChainProxy","lineno":238,"native":false},{"filename":"CompositeFilter.java","function":"doFilter","module":"org.springframework.web.filter.CompositeFilter","lineno":74,"native":false},{"filename":"CompositeFilter.java","function":"doFilter","module":"org.springframework.web.filter.CompositeFilter$VirtualFilterChain","lineno":113,"native":false},{"filename":"HandlerMappingIntrospector.java","function":"lambda$createCacheFilter$4","module":"org.springframework.web.servlet.handler.HandlerMappingIntrospector","lineno":267,"native":false},{"filename":"CompositeFilter.java","function":"doFilter","module":"org.springframework.web.filter.CompositeFilter$VirtualFilterChain","lineno":113,"native":false},{"filename":"FilterChainProxy.java","function":"doFilter","module":"org.springframework.security.web.FilterChainProxy","lineno":191,"native":false},{"filename":"FilterChainProxy.java","function":"doFilterInternal","module":"org.springframework.security.web.FilterChainProxy","lineno":233,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":224,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"lambda$wrap$0","module":"org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation","lineno":323,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"DisableEncodeUrlFilter.java","function":"doFilterInternal","module":"org.springframework.security.web.session.DisableEncodeUrlFilter","lineno":42,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"WebAsyncManagerIntegrationFilter.java","function":"doFilterInternal","module":"org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter","lineno":62,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"SecurityContextHolderFilter.java","function":"doFilter","module":"org.springframework.security.web.context.SecurityContextHolderFilter","lineno":69,"native":false},{"filename":"SecurityContextHolderFilter.java","function":"doFilter","module":"org.springframework.security.web.context.SecurityContextHolderFilter","lineno":82,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"HeaderWriterFilter.java","function":"doFilterInternal","module":"org.springframework.security.web.header.HeaderWriterFilter","lineno":75,"native":false},{"filename":"HeaderWriterFilter.java","function":"doHeadersAfter","module":"org.springframework.security.web.header.HeaderWriterFilter","lineno":90,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"LogoutFilter.java","function":"doFilter","module":"org.springframework.security.web.authentication.logout.LogoutFilter","lineno":93,"native":false},{"filename":"LogoutFilter.java","function":"doFilter","module":"org.springframework.security.web.authentication.logout.LogoutFilter","lineno":107,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"BasicAuthenticationFilter.java","function":"doFilterInternal","module":"org.springframework.security.web.authentication.www.BasicAuthenticationFilter","lineno":213,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"RequestCacheAwareFilter.java","function":"doFilter","module":"org.springframework.security.web.savedrequest.RequestCacheAwareFilter","lineno":63,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"SecurityContextHolderAwareRequestFilter.java","function":"doFilter","module":"org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter","lineno":179,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"AnonymousAuthenticationFilter.java","function":"doFilter","module":"org.springframework.security.web.authentication.AnonymousAuthenticationFilter","lineno":100,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"ExceptionTranslationFilter.java","function":"doFilter","module":"org.springframework.security.web.access.ExceptionTranslationFilter","lineno":120,"native":false},{"filename":"ExceptionTranslationFilter.java","function":"doFilter","module":"org.springframework.security.web.access.ExceptionTranslationFilter","lineno":126,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":137,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":227,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"wrapFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$ObservationFilter","lineno":240,"native":false},{"filename":"AuthorizationFilter.java","function":"doFilter","module":"org.springframework.security.web.access.intercept.AuthorizationFilter","lineno":101,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"doFilter","module":"org.springframework.security.web.ObservationFilterChainDecorator$VirtualFilterChain","lineno":128,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"lambda$wrapSecured$0","module":"org.springframework.security.web.ObservationFilterChainDecorator","lineno":82,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"lambda$wrap$1","module":"org.springframework.security.web.ObservationFilterChainDecorator$AroundFilterObservation$SimpleAroundFilterObservation","lineno":340,"native":false},{"filename":"ObservationFilterChainDecorator.java","function":"lambda$wrap$1","module":"org.springframework.security.web.ObservationFilterChainDecorator$FilterObservation$SimpleFilterObservation","lineno":479,"native":false},{"filename":"FilterChainProxy.java","function":"lambda$doFilterInternal$3","module":"org.springframework.security.web.FilterChainProxy","lineno":231,"native":false},{"filename":"CompositeFilter.java","function":"doFilter","module":"org.springframework.web.filter.CompositeFilter$VirtualFilterChain","lineno":108,"native":false},{"filename":"ApplicationFilterChain.java","function":"doFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":140,"native":false},{"filename":"ApplicationFilterChain.java","function":"internalDoFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":164,"native":false},{"filename":"OncePerRequestFilter.java","function":"doFilter","module":"org.springframework.web.filter.OncePerRequestFilter","lineno":116,"native":false},{"filename":"ApplicationFilterChain.java","function":"doFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":140,"native":false},{"filename":"ApplicationFilterChain.java","function":"internalDoFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":164,"native":false},{"filename":"WsFilter.java","function":"doFilter","module":"org.apache.tomcat.websocket.server.WsFilter","lineno":51,"native":false},{"filename":"ApplicationFilterChain.java","function":"doFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":140,"native":false},{"filename":"ApplicationFilterChain.java","function":"internalDoFilter","module":"org.apache.catalina.core.ApplicationFilterChain","lineno":195,"native":false},{"filename":"HttpServlet.java","function":"service","module":"jakarta.servlet.http.HttpServlet","lineno":658,"native":false},{"filename":"FrameworkServlet.java","function":"service","module":"org.springframework.web.servlet.FrameworkServlet","lineno":885,"native":false},{"filename":"HttpServlet.java","function":"service","module":"jakarta.servlet.http.HttpServlet","lineno":564,"native":false},{"filename":"FrameworkServlet.java","function":"doGet","module":"org.springframework.web.servlet.FrameworkServlet","lineno":903,"native":false},{"filename":"FrameworkServlet.java","function":"processRequest","module":"org.springframework.web.servlet.FrameworkServlet","lineno":1014,"native":false},{"filename":"DispatcherServlet.java","function":"doService","module":"org.springframework.web.servlet.DispatcherServlet","lineno":979,"native":false},{"filename":"DispatcherServlet.java","function":"doDispatch","module":"org.springframework.web.servlet.DispatcherServlet","lineno":1089,"native":false},{"filename":"AbstractHandlerMethodAdapter.java","function":"handle","module":"org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter","lineno":87,"native":false},{"filename":"RequestMappingHandlerAdapter.java","function":"handleInternal","module":"org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter","lineno":891,"native":false},{"filename":"RequestMappingHandlerAdapter.java","function":"invokeHandlerMethod","module":"org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter","lineno":986,"native":false},{"filename":"ServletInvocableHandlerMethod.java","function":"invokeAndHandle","module":"org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod","lineno":118,"native":false},{"filename":"InvocableHandlerMethod.java","function":"invokeForRequest","module":"org.springframework.web.method.support.InvocableHandlerMethod","lineno":191,"native":false},{"filename":"InvocableHandlerMethod.java","function":"doInvoke","module":"org.springframework.web.method.support.InvocableHandlerMethod","lineno":258,"native":false},{"filename":"Method.java","function":"invoke","module":"java.lang.reflect.Method","lineno":569,"native":false},{"filename":"DelegatingMethodAccessorImpl.java","function":"invoke","module":"jdk.internal.reflect.DelegatingMethodAccessorImpl","lineno":43,"native":false},{"filename":"NativeMethodAccessorImpl.java","function":"invoke","module":"jdk.internal.reflect.NativeMethodAccessorImpl","lineno":77,"native":false},{"filename":"NativeMethodAccessorImpl.java","function":"invoke0","module":"jdk.internal.reflect.NativeMethodAccessorImpl","native":true},{"filename":"PersonController.java","function":"person","module":"io.sentry.samples.spring.boot.jakarta.PersonController","lineno":33,"in_app":true,"native":false}]},"mechanism":{"type":"LogbackSentryAppender","exception_id":0}}]},"level":"error","transaction":"GET /person/10","fingerprint":[],"modules":{"spring-webflux":"6.2.8","netty-resolver-dns":"4.1.122.Final","spring-security-config":"6.4.7","log4j-api":"2.24.3","tomcat-embed-core":"10.1.42","netty-buffer":"4.1.122.Final","netty-resolver":"4.1.122.Final","jackson-module-parameter-names":"2.18.4","spring-webmvc":"6.2.8","netty-codec-http":"4.1.122.Final","spring-graphql":"1.3.5","spring-boot-starter-graphql":"3.4.7","netty-resolver-dns-classes-macos":"4.1.122.Final","sentry-quartz":"8.17.0","aspectjweaver":"1.9.24","sentry-spring-jakarta":"8.17.0","micrometer-commons":"1.14.8","spring-boot-starter-aop":"3.4.7","spring-boot-starter-webflux":"3.4.7","tomcat-embed-websocket":"10.1.42","spring-boot-starter-reactor-netty":"3.4.7","spring-messaging":"6.2.8","spring-boot-starter-logging":"3.4.7","jakarta.annotation-api":"2.1.1","reactor-core":"3.7.7","jackson-core":"2.18.4.1","sentry-graphql-22":"8.17.0","jackson-datatype-jdk8":"2.18.4","kotlin-stdlib":"1.9.24","spring-core":"6.2.8","netty-codec-dns":"4.1.122.Final","reactor-netty-http":"1.2.7","spring-boot-actuator-autoconfigure":"3.4.7","spring-aop":"6.2.8","kotlin-stdlib-jdk8":"1.9.24","spring-security-core":"6.4.7","kotlin-stdlib-jdk7":"1.9.24","netty-transport-native-unix-common":"4.1.122.Final","spring-beans":"6.2.8","logback-classic":"1.5.18","tomcat-embed-el":"10.1.42","hsqldb":"2.6.1","jackson-databind":"2.18.4","HdrHistogram":"2.2.2","netty-transport-native-epoll":"4.1.122.Final-linux-x86_64","sentry-spring-boot-jakarta":"8.17.0","spring-boot-starter-actuator":"3.4.7","reactor-netty-core":"1.2.7","p6spy":"3.9.1","sentry-logback":"8.17.0","spring-boot-starter-web":"3.4.7","context-propagation":"1.1.3","logback-core":"1.5.18","spring-tx":"6.2.8","netty-transport-classes-epoll":"4.1.122.Final","spring-boot-starter-security":"3.4.7","spring-security-web":"6.4.7","HikariCP":"5.1.0","annotations":"13.0","spring-jcl":"6.2.8","java-dataloader":"3.3.0","spring-boot-starter-json":"3.4.7","spring-web":"6.2.8","micrometer-core":"1.14.8","spring-boot-starter":"3.4.7","netty-transport":"4.1.122.Final","sentry-spring-boot-starter-jakarta":"8.17.0","netty-codec-socks":"4.1.122.Final","spring-expression":"6.2.8","spring-boot-starter-jdbc":"3.4.7","spring-websocket":"6.2.8","graphql-java":"22.3","spring-boot-autoconfigure":"3.4.7","spring-boot-starter-websocket":"3.4.7","LatencyUtils":"2.0.3","log4j-to-slf4j":"2.24.3","micrometer-observation":"1.14.8","spring-context":"6.2.8","spring-jdbc":"6.2.8","kotlin-reflect":"1.9.24","spring-boot":"3.4.7","spring-boot-starter-quartz":"3.4.7","jackson-annotations":"2.18.4","spring-boot-starter-tomcat":"3.4.7","netty-common":"4.1.122.Final","slf4j-api":"2.0.17","spring-boot-actuator":"3.4.7","sentry-graphql-core":"8.17.0","micrometer-jakarta9":"1.14.8","reactive-streams":"1.0.4","jul-to-slf4j":"2.0.17","netty-handler":"4.1.122.Final","sentry":"8.17.0","netty-resolver-dns-native-macos":"4.1.122.Final-osx-x86_64","jackson-datatype-jsr310":"2.18.4","sentry-reactor":"8.17.0","netty-codec":"4.1.122.Final","quartz":"2.3.2","netty-handler-proxy":"4.1.122.Final","spring-context-support":"6.2.8","netty-codec-http2":"4.1.122.Final","spring-security-crypto":"6.4.7","sentry-jdbc":"8.17.0","snakeyaml":"2.3","mchange-commons-java":"0.2.15"},"event_id":"543d49a549724ce4918e60b542dc03cc","contexts":{"runtime":{"name":"Spring Boot","version":"3.4.7"},"spring":{"active_profiles":[]},"trace":{"trace_id":"eb2d9f596b7a492facff1e3e5a8c9ec1","span_id":"782369fc14484227","parent_span_id":"c0a6e27836714ddf","op":"spanCreatedThroughSentryApi","origin":"manual","data":{"thread.name":"http-nio-8080-exec-1","thread.id":"60"}}},"sdk":{"name":"sentry.java.spring-boot.jakarta","version":"8.17.0","packages":[{"name":"maven:io.sentry:sentry","version":"8.17.0"},{"name":"maven:io.sentry:sentry-spring-boot-starter-jakarta","version":"8.17.0"},{"name":"maven:io.sentry:sentry-logback","version":"8.17.0"},{"name":"maven:io.sentry:sentry-graphql-22","version":"8.17.0"},{"name":"maven:io.sentry:sentry-quartz","version":"8.17.0"},{"name":"maven:io.sentry:sentry-jdbc","version":"8.17.0"},{"name":"maven:io.sentry:sentry-spring-boot-jakarta","version":"8.17.0"},{"name":"maven:io.sentry:sentry-graphql-core","version":"8.17.0"},{"name":"maven:io.sentry:sentry-spring-jakarta","version":"8.17.0"},{"name":"maven:io.sentry:sentry-reactor","version":"8.17.0"}],"integrations":["SpringBoot3","UncaughtExceptionHandler","ShutdownHook","Spotlight","JDBC","Spring6GrahQLWebMVC","GraphQL-v22","Quartz","Logback"]},"request":{"url":"http://localhost:8080/person/10","method":"GET","query_string":"q=123456&b=X","cookies":"JSESSIONID=[Filtered]","headers":{"authorization":"Basic dXNlcjpwYXNzd29yZA==","cookie":"JSESSIONID=[Filtered]","host":"localhost:8080","content-type":"application/json","accept-encoding":"br, deflate, gzip, x-gzip","user-agent":"IntelliJ HTTP Client/IntelliJ IDEA 2025.1.2","accept":"*/*"}},"environment":"production","platform":"java","user":{"username":"user","ip_address":"127.0.0.1"},"server_name":"localhost","breadcrumbs":[{"timestamp":"2025-07-21T14:01:35.922Z","message":"Started SentryDemoApplication in 1.659 seconds (process running for 1.774)","data":{},"category":"io.sentry.samples.spring.boot.jakarta.SentryDemoApplication","level":"info"},{"timestamp":"2025-07-21T14:02:14.439Z","message":"Initializing Spring DispatcherServlet 'dispatcherServlet'","data":{},"category":"org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/]","level":"info"},{"timestamp":"2025-07-21T14:02:14.439Z","message":"Initializing Servlet 'dispatcherServlet'","data":{},"category":"org.springframework.web.servlet.DispatcherServlet","level":"info"},{"timestamp":"2025-07-21T14:02:14.440Z","message":"Completed initialization in 1 ms","data":{},"category":"org.springframework.web.servlet.DispatcherServlet","level":"info"},{"timestamp":"2025-07-21T14:02:14.441Z","type":"http","data":{"method":"GET","url":"/person/10"},"category":"http"}],"extra":{"thread_name":"http-nio-8080-exec-1"}}

adinauer avatar Jul 21 '25 14:07 adinauer

@adinauer I put some context to the DEBUG: Serializing object: you mentioned below:

  • NOT WORKING 
bash-3.2$ grep "DEBUG: " 20250721_not_working.txt

17:00:30,903 |-WARN in io.sentry.logback.SentryAppender[Sentry] - Failed to init Sentry during appender initialization: DSN is required. Use empty string or set enabled to false in SentryOptions to disable SDK.

INFO: Initializing SDK with DSN: 'https://xxxxxxx.ingest.us.sentry.io/yyyyyyyy'
INFO: No outbox dir path is defined in options.
INFO: No sentry-debug-meta.properties file was found.
DEBUG: UncaughtExceptionHandlerIntegration enabled: true
DEBUG: UncaughtExceptionHandlerIntegration installed.
DEBUG: ShutdownHookIntegration installed.
DEBUG: SpotlightIntegration is not enabled. BeforeEnvelopeCallback is already set or spotlight is not enabled.
DEBUG: Using openTelemetryMode AUTO
DEBUG: Using span factory io.sentry.DefaultSpanFactory
DEBUG: Using scopes storage io.sentry.DefaultScopesStorage
INFO: Cache dir is not set, not finalizing the previous session.
DEBUG: Capturing event: 2c6dfe8d9f6c484d824b025d67c3e9e3
INFO: sentry-external-modules.txt file was not found.
INFO: Session is null on scope.withSession
DEBUG: Serializing object: {
        "timestamp": "2025-07-21T15:00:34.321Z",
        "message": {
                "formatted": "Unexpected error occurred in scheduled task",
                "message": "Unexpected error occurred in scheduled task"
        },
        "logger": "org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler",
        "exception": {
                "values": [
                        {
                                "type": "DynamoDbException",
                                "value": "Service 'dynamodb' is not enabled. Please check your 'SERVICES' configuration variable. (Service: DynamoDb, Status Code: 501, Request ID: a637fe6d-c653-4a88-a2b8-0f88d8fa24ad) (SDK Attempt Count: 1)",
                                "module": "software.amazon.awssdk.services.dynamodb.model",
                                "thread_id": 91,...nn

DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed
DEBUG: Returning no-op for instrumenter SENTRY as the SDK has been configured to use instrumenter OTEL
2025-07-21T17:01:20.665+02:00 ERROR 25694 --- [acme-application] [omcat-handler-0] d.a.a.i.inbound.web.OtherCompanyApiController     : Testing Sentry integration, do not use in production.
DEBUG: Capturing event: d0a09e0807d548a78d51201c3dc722b0
INFO: Session is null on scope.withSession
DEBUG: Serializing object: {
        "timestamp": "2025-07-21T15:01:20.665Z",
        "message": {
                "formatted": "Testing Sentry integration, do not use in production.",
                "message": "Testing Sentry integration, do not use in production."
        },              
        "logger": "de.acme.acme_application.infrastructure.inbound.web.OtherCompanyApiController",
        "threads": { 
                "values": [     
                        {       
                                "id": 122,
                                "priority": 5,
                                "name": "tomcat-handler-0",
                                "state": "RUNNABLE",
                                "crashed": true,
                                "daemon": true,

DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed
  • NOT WORKING - Same source code and build, just waited ~45 minutes
bash-3.2$ grep "DEBUG: " 20250721_working.txt

17:45:55,937 |-WARN in io.sentry.logback.SentryAppender[Sentry] - Failed to init Sentry during appender initialization: DSN is required. Use empty string or set enabled to false in SentryOptions to disable SDK.
INFO: Initializing SDK with DSN: 'https://xxxxxxxx.ingest.us.sentry.io/yyyyyyy'
INFO: No outbox dir path is defined in options.
INFO: No sentry-debug-meta.properties file was found.
DEBUG: UncaughtExceptionHandlerIntegration enabled: true
DEBUG: UncaughtExceptionHandlerIntegration installed.
DEBUG: ShutdownHookIntegration installed.
DEBUG: SpotlightIntegration is not enabled. BeforeEnvelopeCallback is already set or spotlight is not enabled.
DEBUG: Using openTelemetryMode AUTO
DEBUG: Using span factory io.sentry.DefaultSpanFactory
DEBUG: Using scopes storage io.sentry.DefaultScopesStorage
DEBUG: Capturing event: 7c979ebe6d5d4685858a0dd60ecdb864
INFO: sentry-external-modules.txt file was not found.
INFO: Session is null on scope.withSession
DEBUG: Serializing object: {
        "timestamp": "2025-07-21T15:45:59.524Z",
        "message": {
                "formatted": "Unexpected error occurred in scheduled task",
                "message": "Unexpected error occurred in scheduled task"
        },
        "logger": "org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler",
        "exception": {
                "values": [
                        {
                                "type": "DynamoDbException",
                                "value": "Service 'dynamodb' is not enabled. Please check your 'SERVICES' configuration variable. (Service: DynamoDb, Status Code: 501, Request ID: d664b238-5a74-47ab-acc1-f5435e5beeaf) (SDK Attempt Count: 1)",
                                "module": "software.amazon.awssdk.services.dynamodb.model",
                                "thread_id": 91,...
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed
DEBUG: Returning no-op for instrumenter SENTRY as the SDK has been configured to use instrumenter OTEL
DEBUG: Capturing event: eca1d265bc974fbfbfcc3b4373d4257d
INFO: Session is null on scope.withSession
DEBUG: Serializing object: {
        "timestamp": "2025-07-21T15:46:05.801Z",
        "message": {
                "formatted": "Testing Sentry integration, do not use in production.",
                "message": "Testing Sentry integration, do not use in production."
        },...
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed

According to diff both logs are pretty much the same: Timestamps and a bunch of ids differ, of course. I suspect that Sentry.io reacts differently to events from the 8.0.17 client after the application restarted or disconnected for a certain amount of time.

jrs666666 avatar Jul 21 '25 15:07 jrs666666

@jrs666666 can you please look at Settings | Stats & Usage in Sentry UI please. It might show you why events are not making it through. If there's no helpful data there or you'd prefer we take a look you may also send me an email to [email protected] . Just send a link to this GH issue and provide your org ID and project name.

adinauer avatar Jul 22 '25 08:07 adinauer

@adinauer Looking into Settings >> ACME >> Stats >> Stats & Usage provided this clue:

Image 
DEBUG: Capturing event: 3c179ba4a1714b9a925a2e7119ef977a
INFO: sentry-external-modules.txt file was not found.
INFO: Session is null on scope.withSession
DEBUG: Serializing object: {
	"timestamp": "2025-07-22T14:26:39.568Z",
	"message": {
		"formatted": "Unexpected error occurred in scheduled task",
		"message": "Unexpected error occurred in scheduled task"
...
}
DEBUG: Envelope sent successfully.
DEBUG: Envelope flushed

It seems that the SDK doesn't report if an event gets dropped. The X-Sentry-Rate-Limit-* headers (if any) in the response seem not translated or recognized from the SDK.

jrs666666 avatar Jul 22 '25 14:07 jrs666666

Since we resolved the problem via email, I'll now close the issue.

adinauer avatar Jul 30 '25 08:07 adinauer