Add optional multi-cluster support for Grafana dashboards

[pmcgrath]

What problem are you trying to solve?

We use thanos to centralise all our metrics in a single cluster, but the dashboards in grafana/dashboards are not multi-cluster aware

It would be nice if they could support this as an optional, so where multiple clusters are detected it could display a cluster drop down and if not detected would ignore

How should the problem be solved?

I can create a PR for this if there are no objections

Any alternatives you've considered?

We currently cater for this with our own custom chart where we pre-process these files injecting this option into each of the dashboard files in grafana/dashboards

• Inject an additional cluster template variable as the first drop down (Only visible if multiple clusters detected)
• Alter all the dashboard queries to filter based on the cluster using a regex

We are replicating how they do this in kube-prometheus-stack

• Template variable
• Query with cluster filter

Downside of this is that we need to watch/check for changes when these dashboards change and re-run this pre-processing script so we have updated versions that are multi-cluster aware

How would users interact with this feature?

Interaction is via Grafana

• If multiple clusters detected then a cluster drop down would appear at the top left of the dashboard which would allow filtering by cluster.
• If only a single cluster detected (default) then the dashboard would appear as they do now, the queries would use the cluster but using a regex which would match all

Would you like to work on this feature?

maybe

[adleong]

Hi @pmcgrath! A multi-cluster focused grafana dashboard sounds hugely useful, I love this idea. The nice thing is that grafana dashboards can be built and shared externally and this shouldn't require any changes inside the Linkerd repo itself. I'm looking forward to seeing what you come up with if you decide to pick this up.

[pmcgrath]

Best to wait on this PR before I proceed with this - https://github.com/linkerd/linkerd2/issues/10873

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[kflynn]

@cpotteck has expressed interest in working on this! 🙂

[cpotteck]

Hi @pmcgrath, I asked for this issue to be re-opened as we are in the same situation as you and have to implement a workaround. I could work on this if necessary and once I have some time.

You did mention you would like to create a PR for it so checking in with you first to see if you are still interested, and if you've changed the workaround implementation on your side?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[dsiperek-vendavo]

Any updates on this?

[cpotteck]

@dsiperek-vendavo we are still interested in seeing this implemented, though I have unfortunately not had time to work on it myself so far. Our focus when it comes to Linkerd has recently been on implementing Edge releases so I can't really give a timeline on my end.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.