fluxcd
Support cross cloud federation
GCP supports authenticating from different trusted identities. One possible authentication story is a user wants to auth from AWS against a GCloud Container Registry. Currently the auth code only supports querying the internal metadata url via the const GCP_TOKEN_URL.
References: https://cloud.google.com/iam/docs/configuring-workload-identity-federation https://cloud.google.com/iam/docs/using-workload-identity-federation#aws_3
I'm going to look into taking this on. I had assumed that the GCP provider operated on the standard GCP auth resolution, but as it works here you have to either pass a secret into the upstream resource (e.g. OCIRepository), or you have to be running in GCP and using a metadata endpoint for auth. I run in all 3 clouds, and wanted to distribute all artifacts from a centralized place in GCP, and I was really trying to avoid shipping service account json creds around everywhere. I don't want to use node-level permissions because that requires allowing a pod to access node permissions, which we lock down by default, and we use a custom CNI in GKE which doesn't play well with the GKE Metadata Server daemon, so that's also not an option.
This will be supported at the controller level starting with Flux 2.6 for some APIs, then for all APIs in Flux 2.7. Track progress here: https://github.com/fluxcd/flux2/issues/5022
In the future we may support object-level as well.
Supporting this for object-level workload identity is easy: Among the three cloud providers only GCP needs changes in our code.
The auth.Provider.GetAudience() interface needs to receive a serviceAccount parameter. In the GCP provider we look up a custom Flux annotation in this service account, e.g. gcp.auth.fluxcd.io/workloadIdentityProvider, containing the full resource name of a GCP Workload Identity Provider. This is the required audience for issuing the Kubernetes service account token, and also the audience required for exchanging the Kubernetes service account token for a GCP access token.