Matt Boveri

I have the JSON, but am getting stuck on the volume mounting bit, do you have an example of that I could take a look at?

I was able to get that working but am now getting the following error in the sidecar's container logs: `level=warn ts=2020-08-26T21:17:02.122Z caller=queue_manager.go:534 component=queue_manager msg="Unrecoverable error sending samples to remote storage"...

For anyone else attempting to mount volumes to Prometheus, the minimum version is 8.13.13 when Volume and VolumeMounts were added - https://github.com/helm/charts/commit/ef0d749132ecfa61b2ea47ccacafeaf5cf1d3d77

Looks like setting the follow did the trick. We are working on getting some better filtering, so have stopped sending for now, but we were able to get the OpenStack...

We will also still need to figure out why also, we need to figure out why some metrics are getting rejected **we see errors like:** `level=warn ts=2020-08-27T23:42:27.611Z caller=queue_manager.go:534 component=queue_manager msg="Unrecoverable...

I think at least: `level=warn ts=2020-08-27T23:23:07.075Z caller=queue_manager.go:534 component=queue_manager msg="Unrecoverable error sending samples to remote storage" err="rpc error: code = InvalidArgument desc = One or more TimeSeries could not be written:...

Awesome, thanks @Dnefedkin !

Would this also add functionality to allow the basic auth filter to be able to consume from a Kubernetes secret or environment variable generated by a Kubernetes secret?

That would be great!