Add opentelemetry instrumentation

[hamersaw]

TL;DR

Adding a new telemetryutils package that adapts the opentelemetry functionality for use in Flyte repos. This includes support for multiple TracerProviders allowing logical partitioning of tracers within single binary. Also instruments the blobstore and provides a wrapper for instrumenting k8s client.

Type

• [ ] Bug Fix
• [x] Feature
• [ ] Plugin

Are all requirements met?

• [x] Code completed
• [x] Smoke tested
• [ ] Unit tests added
• [ ] Code documentation added
• [ ] Any pending items have an associated Issue

Complete description

^^^

Tracking Issue

https://github.com/flyteorg/flyte/issues/3304

Follow-up issue

NA

[codecov[bot]]

Codecov Report

Merging #146 (9cbeef9) into master (3322b07) will decrease coverage by 1.44%. The diff coverage is 40.20%.

@@            Coverage Diff             @@
##           master     #146      +/-   ##
==========================================
- Coverage   68.18%   66.75%   -1.44%     
==========================================
  Files          69       73       +4     
  Lines        4083     4268     +185     
==========================================
+ Hits         2784     2849      +65     
- Misses       1141     1249     +108     
- Partials      158      170      +12     

Flag	Coverage Δ
unittests	66.75% <40.20%> (-1.44%)	:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
cli/pflags/api/namedtype_enumer.go	0.00% <ø> (ø)
storage/config.go	66.66% <ø> (ø)
storage/storage.go	100.00% <ø> (ø)
storage/utils.go	65.00% <0.00%> (-5.28%)	:arrow_down:
telemetryutils/config.go	0.00% <0.00%> (ø)
telemetryutils/k8s.go	0.00% <0.00%> (ø)
telemetryutils/config_flags.go	25.00% <25.00%> (ø)
storage/stow_store.go	75.70% <31.81%> (-3.85%)	:arrow_down:
telemetryutils/factory.go	64.51% <64.51%> (ø)
storage/mem_store.go	82.00% <81.81%> (-0.93%)	:arrow_down:
... and 3 more

... and 1 file with indirect coverage changes

[hamersaw]

closed in favor of https://github.com/flyteorg/flytestdlib/pull/149

[kumare3]

Does it also change are Dependency from Prometheus to Open telemetry? Should we