clickhouse-operator icon indicating copy to clipboard operation
clickhouse-operator copied to clipboard

Metrics suddenly stopped

Open jgrossmac opened this issue 3 years ago • 1 comments

Ran a curl against the operator on port 8888, then did it a few minutes later and no metrics.

Noticed this in the logs of the metrics-exporter. Any ideas? Seems like an old design decision popped it head back up?

I0217 01:40:19.641319 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter I0217 01:40:19.641426 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter I0217 01:40:19.641448 1 exporter.go:92] Removed ClickHouseInstallation (simple-01/clickhouse) from Exporter I0217 01:40:19.641496 1 exporter.go:92] Removed ClickHouseInstallation (pv-log/clickhouse) from Exporter I0217 01:40:19.641534 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter I0217 01:40:19.641585 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter I0217 01:40:19.641626 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter I0217 01:40:19.641678 1 exporter.go:92] Removed ClickHouseInstallation (data-team1/clickhouse) from Exporter

jgrossmac avatar Feb 17 '22 02:02 jgrossmac

No, described issue is not actual now, currently metrics-exporter will write chi_clickhouse_metrics_fetch_errors if fetch fail could you share kubectl get chi --all-namespces?

Which clickhouse-operator version do you use? could you share kubectl get pods --all-namespaces -l app=clickhouse-operator ?

Looks like you delete your chi manifest or delete CRDs kubectl get crds | grep clickhouse

Slach avatar Feb 17 '22 05:02 Slach