OTEL setup via environment variables

[stephanos]

What changed?

• allow configuring a custom OTEL service.name prefix
• allow configuring an OTEL exporter via env vars (following the OTEL spec)

It is separate from configuring Server's OTEL behavior via the config file as this uses environment variables - which is how OTEL is usually configured. Let me know if there are concerns with that.

TODO: If this is accepted, I'll update the docs, too.

Why?

Addresses:

• https://github.com/temporalio/temporal/issues/4042
• https://github.com/temporalio/temporal/issues/4041
• our own (increasing) need to enable OpenTelemetry tracing without touching the Server config (for ad-hoc debugging when working on a SDK, for example)

How did you test it?

I ran an OTEL collector to verify it sends the tracing exports.

These are the env vars I set to make it work:

// these just make the batch processor flush immediately
os.Setenv("OTEL_BSP_SCHEDULE_DELAY", "0")
os.Setenv("OTEL_BSP_MAX_QUEUE_SIZE", "0")

os.Setenv("OTEL_SERVICE_NAME", "myservice")
os.Setenv("OTEL_TRACES_EXPORTER", "oltp")
os.Setenv("OTEL_EXPORTER_OTLP_TRACES_ENDPOINT", "http://localhost:4317")

Potential risks

Accidentally turning on OTEL - but it seems highly unlikely given the very specific environment variables. In that case, it crashes immediately if there is no OTEL collector listening on the port.

Is hotfix candidate?

No