vector icon indicating copy to clipboard operation
vector copied to clipboard

Document that metric sinks aggregate metrics within a batch

Open bruceg opened this issue 4 years ago • 3 comments

Most of the metric sinks aggregate metrics within a batch. That is, incremental metrics are added together and absolute metrics are replaced by the later value. As noted by https://github.com/timberio/vector/pull/7846#pullrequestreview-686985284 this is not adequately described in the documentation for those sinks. This applies to the following sinks:

  • aws_cloudwatch_metrics
  • datadog_metrics
  • stackdriver_metrics
  • influxdb_metrics
  • prometheus_exporter
  • prometheus_remote_write
  • sematext_metrics

bruceg avatar Jun 18 '21 23:06 bruceg

I was tripped up by this when naively assuming I could import a backlog of metrics from CSV files to influxdb.

My workaround is specifying a batch.max_events: 1, which is terrible for performance, of course.

I my opinion, this aggregation should not occur in the first place, but that is a different discussion, I guess.

moschroe avatar Oct 24 '23 09:10 moschroe

Note stackdriver_metrics no longer aggregates within a batch.

StephenWakely avatar Oct 25 '23 13:10 StephenWakely

How long before influxdb_metrics can support turning off auto-aggregation?

curtiseng avatar Jun 24 '24 11:06 curtiseng