feat(otel): add metrics about loaded certificates
It was previously difficult to determine if a certinel implementation was properly detecting changes and reloading certificates. This lack of observability has made some teams uneasy of relying on certinel watchers.
This changeset adds OpenTelemetry metrics support to the fswatcher and ticker packages. Both packages have been configures to report "certificate.not_before_timestamp" and "certificate.not_after_timestamp" meters, with attributes "certificate.serial" and "certificate.path". While OpenTelemetry defines Semanatic Conventions for TLS, the current conventions are scoped for client or server certificates, which this package is unable to determine.
Fixes: #27
I've implemented the metrics in two different ways, one synchronous and one asynchronous, as I wasn't sure which is preferable. The former updates the metric when we reload from disk, even if nothing is collecting. Whereas the former only updates the metric if something is collecting, and returns data for the certificate that would be returned.
It seems more obvious and straightforward to me to just use the synchronous approach in loadCertificate. The watcher takes an explicit action on the certificate and thusly the metric is updated. I think the simpler tests attest to this as well.
Answering my own question: using "observable" ensures only one series is exported at a time per certificate path, otherwise older series continue to be exported for some time. This would have been caught by the ticker tests, but the "simplier" tests were actually wrong and not properly testing the behavior.
I'm not happy with this implementation.