[Epic] Hydration and rehydration latency metrics per dataflow

[bosconi]

Product outcome

Metrics are published that allow internal dashboards to report per-dataflow hydration and rehydration latency, potentially along with normalized metrics such as rate (amount of data per unit time).

Discovery

Per the MVP metrics section of the Document and Measure Sources of Unavailability product spec, the proposed MVP metrics are:

• Hydration time per dataflow.
• Rehydration time per dataflow.

hydration time is apparently already measured. Example Grafana panel.

Initial questions:

• Is the current hydration time metric sufficient for the hydration time ask? (Is it per dataflow?)
• For both hydration and rehydration, would it be useful to publish a normalized rate metric (the time per unit of data ingested)?

Work items

- [ ] #1

Decision log

• 6 May 2024. Issue created.