feat: OpenTelemetry Protocol support for metrics

[Dubzer]

hii i've implemented the support for OTLP!

protocol this version only supports HTTP Protobuf because i think it is the most widely supported, including Prometheus. support for HTTP JSON and gRPC can be added later if necessary

about the implementation at first, i wanted to use OTEL library as a single source of truth for metrics. unfortunately, despite OTEL having a separate spec for converting its metrics to the Prometheus format, the rust library was deprecated for some reason. so, i decided to create a small abstraction on the exporters, and now the metrics code lives in src/metrics

todo please note that the opentelemetry client uses reqwest, but i'm unsure if it needs to be configured specifically for this project. also, please verify that the feature setup is correct.

i hope i didn't forget anything else

fixes: #1458

[aawsome]

Hi @Dubzer - thanks a lot for the PR!

I just took a short look right now, but this already looks very good!

The features setup looks good, but you are right, we need to take a good look about the reqwest dependency. Do you know if request features can be specified as opentelemetry crate features? Or if there is way to use the library to just create the payload such that can call reqwest ourselves?

I'll take a bit of time soon for a deeper review!

[Dubzer]

Do you know if request features can be specified as opentelemetry crate features?

there are some features related to the reqwest:

• reqwest-blocking-client: Use reqwest blocking http client. This feature is enabled by default.
• reqwest-client: Use reqwest http client.
• reqwest-rustls: Use reqwest with TLS with system trust roots via rustls-native-certs crate.
• reqwest-rustls-webpki-roots: Use reqwest with TLS with Mozilla’s trust roots via webpki-roots crate.

Or if there is way to use the library to just create the payload such that can call reqwest ourselves

i think it is possible to provide our own HTTP client via with_http_client; the implementation of the HttpClient trait for the reqwest seems to be pretty straightforward

[aawsome]

Or if there is way to use the library to just create the payload such that can call reqwest ourselves

i think it is possible to provide our own HTTP client via with_http_client; the implementation of the HttpClient trait for the reqwest seems to be pretty straightforward

As we do this already for the prometheus push client implementation, I'd vote for this direction! I agree that this should be pretty easy using with_http_client :+1:

[Dubzer]

@aawsome hmm.. the opentelemetry-rust has Apache 2.0 licensing, so i'm not sure if it's appropriate to use their code in this repository

and also, it seems that if we enable reqwest-rustls for the opentelemetry-otlp crate, the feature set won't be different than specified for reqwest in our Cargo.toml

i've checked this by running

cargo tree -i reqwest -e features

i think it's fine to stick to their reqwest dependency. what are your thoughts on this?

[aawsome]

and also, it seems that if we enable reqwest-rustls for the opentelemetry-otlp crate, the feature set won't be different than specified for reqwest in our Cargo.toml

Yes, you are right. I'm just a bit afraid of changes in reqwestdependencies will become even more complicated (in case we need that). But we can also deal with this when its time to deal with ;-)

[aawsome]

Thanks a lot @Dubzer for this PR and sorry for the delay!

About prometheus support I wonder if we should just remove this and ask our users to use OTLP to push metrics to prometheus. What do you think?

[Dubzer]

@aawsome well, based on how it was implemented, i don't foresee any maintainability issues. it can help users migrate from restic by reusing their existing infrastructure. it's also easier to setup auth with prometheus. imo, it's fine to keep it. though OTLP may be listed as a better choice in the docs

[aawsome]

Thanks for the feedback @Dubzer. So, for now, we just keep it and see how it evolves with prometheus in the coming months/years! ;-)