`Datadog.tracer.active_correlation` broken in recent release

[mbarany]

We followed the docs here and are using the following in our logger to get the correlation data:

{ dd: ::Datadog.tracer.active_correlation.to_h }

This used to get us the following output:

"dd": {
  "trace_id": <redacted>,
  "span_id": <redacted>,
  "env": null,
  "service": "my-fancy-service",
  "version": null
}

Now we get this output:

"dd": {
  "trace_id": <redacted>,
  "span_id": <redacted>,
  "env": null,
  "service": "rails",
  "version": null
}

We started noticing this in release 0.51.0

[mbarany]

It might be related to this commit: https://github.com/DataDog/dd-trace-rb/commit/91564dafbbfd10b61ab380e52e37b16cf54c38fc

[mbarany]

We got it working by adding the "unified service tagging" ENV vars (DD_SERVICE, DD_ENV, DD_VERSION)

However this is not ideal since we now have to make sure all of our services add the unified service tagging before upgrading to 0.51.0

[mbarany]

Side note: I noticed a few "breaking" changes in the CHANGELOG. Consider using Semantic Versioning - it can help communicate to the consumer the risks of upgrading. DOn't be afraid to bump that major version number 😄

[marcotc]

Hi @mbarany, thank you for reporting this issue.

I tried to reproduce the issue you described, the change in service when invoking Datadog.tracer.active_correlation.to_h, but I can't quite get the same results. I tried with both 0.51.0 and an older version, 0.46.0.

Could you let me know how you are/were setting up the value for my-fancy-service in your service?

I tried a few combinations on changing values that could affect the service name (c.service = 'my-fancy-service' and explicitly setting the service name in the currently active span) but I can't get the same results. (export DD_SERVICE=my-fancy-service also worked, but you already mentioned that you were not originally using this avenue.)

[mbarany]

This is our configuration:

Datadog.configure do |c|
  service_name = 'my-fancy-service'

  c.logger = CustomLogger.create(level: :error)

  c.use(:rails, service_name: service_name)
  c.use(:http, service_name: "#{service_name}-http")
  c.use(:rake, service_name: "#{service_name}-rake")
  c.use(:redis, service_name: "#{service_name}-redis")
  c.use(:active_record, orm_service_name: "#{service_name}-active-record")
  c.use(:sidekiq, service_name: "#{service_name}-sidekiq", client_service_name: "#{service_name}-sidekiq-client")
end

[jonfreeland]

Thank you for reporting @mbarany. We are running into this issue as well and can consistently reproduce. We are setting service_name in code, not via an environment variable. We have pinned at 0.50.0 for the time being.

I agree on the suspect commit, which is part of this PR.

Datadog.configure do |config|
  config.tracer.enabled = Rails.env.production?
  config.logger.level = ::Logger::ERROR
  config.env = ENV["the_env"]
  config.use(:rails, service_name: "the_service_name")
  config.use(:aws)
  config.use(:sidekiq, service_name: "the_service_name-sidekiq", client_service_name: "the_service_name-sidekiq-client")
end

With the above configuration, on 0.50.0, we get "the_service_name" as our service name in Datadog. If we upgrade to 0.51.1, with no other changes, the service name becomes "rails".

[marcotc]

Thank you for the additional data point, @jonfreeland! This is super helpful to the investigation, alongside the info provided by @mbarany.

[marcotc]

👋 @mbarany and @jonfreeland, sorry for the super long wait.

With the release of ddtrace v1.0.0 we've rewritten most of the logic around how the active correlation works.

If you had the change to upgrade, would you know if the correlation API works correctly now for you use case?

From our Upgrade Guide, the method called to retrieve the active correlation has been moved:

Datadog::Tracing.log_correlation, optimized for inline log correlation injection.

Datadog::Tracing.correlation, a genetic correlation data structure.