integrations-core icon indicating copy to clipboard operation
integrations-core copied to clipboard

[Envoy] Support `envoy.cluster.outlier_detection.*` in OpenMetrics V2

Open keisku opened this issue 3 years ago • 2 comments

What does this PR do?

Support envoy.cluster.outlier_detection.* in OpenMetrics V2, /stats/prometheus.

Motivation

Agent doesn't collect envoy.cluster.outlier_detection.* for now even Agent used to collect those metrics from the /stats/prometheus. If my understanding is correct, this behavior changed is from https://github.com/DataDog/integrations-core/pull/10752

Example log message that shows Agent ignore envoy.cluster.outlier_detection.* in OpenMetrics V2.

2022-04-19 04:39:53 UTC | CORE | DEBUG | (pkg/collector/python/datadog_agent.go:128 in LogMessage) | envoy:13076b6152ead410 | (transform.py:80) | Skipping metric `envoy_cluster_outlier_detection_ejections_detected_local_origin_success_rate` as it is not defined in `metrics`

Additional Notes

Review checklist (to be filled by reviewers)

  • [ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • [ ] PR title must be written as a CHANGELOG entry (see why)
  • [ ] Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • [ ] PR must have changelog/ and integration/ labels attached

keisku avatar Apr 20 '22 02:04 keisku

Where can I add test values for envoy.cluster.outlier_detection.* metrics? Maybe, I should add test data to https://github.com/DataDog/integrations-core/blob/8386537099f5497a05192b97e69d97d47e316f78/datadog_checks_base/tests/base/checks/openmetrics/test_interface.py

I want to fix this CI error.

AssertionError: Needed at least 1 candidates for 'envoy.cluster.outlier_detection.ejections_enforced_total', got 0
  Expected:
          MetricStub(name='envoy.cluster.outlier_detection.ejections_enforced_total', type=None, value=None, tags=None, hostname=None, device=None, flush_first_value=None)
  Similar submitted:
  Score   Most similar
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:xds_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:xds_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service2'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service2'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service1'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service1'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:dummy_dynamic_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.cluster.upstream_cx_tx_bytes_total', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:dummy_dynamic_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.server.total_connections', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.56    MetricStub(name='envoy.server.total_connections', type=0, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.54    MetricStub(name='envoy.cluster.upstream_rq_max_duration_reached.count', type=3, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:xds_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=True)
  0.54    MetricStub(name='envoy.cluster.upstream_rq_max_duration_reached.count', type=3, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:xds_cluster'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.54    MetricStub(name='envoy.cluster.upstream_rq_max_duration_reached.count', type=3, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service2'], hostname='stubbed.hostname', device=None, flush_first_value=True)
  0.54    MetricStub(name='envoy.cluster.upstream_rq_max_duration_reached.count', type=3, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service2'], hostname='stubbed.hostname', device=None, flush_first_value=False)
  0.54    MetricStub(name='envoy.cluster.upstream_rq_max_duration_reached.count', type=3, value=0.0, tags=['endpoint:http://localhost:8001/stats/prometheus', 'envoy_cluster:service1'], hostname='stubbed.hostname', device=None, flush_first_value=True)
assert False

keisku avatar Apr 20 '22 08:04 keisku

Codecov Report

Merging #11860 (8c6a02b) into master (8b4acc6) will not change coverage. The diff coverage is n/a.

Flag Coverage Δ
envoy 94.23% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

codecov[bot] avatar Aug 09 '22 08:08 codecov[bot]