Missing support for AWS Workload Federation

[weaversam8]

Hi there! I tried to set up AWS Workload Federation with goth today, and it looks like Goth.Token.subject_token_from_credential_source/2 is missing a clause for AWS tokens. The current clauses are:

defp subject_token_from_credential_source(%{"url" => url, "headers" => headers, "format" => format}, config), do: ...
defp subject_token_from_credential_source(%{"file" => file, "format" => format}, _config), do: ...
defp subject_token_from_credential_source(%{"file" => file}, _config), do: ...

but AWS tokens are stored like this in the GCP credentials.json format:

{
    "type": "external_account",
    "universe_domain": "googleapis.com",
    "audience": "//iam.googleapis.com/projects/XXXXXXXXXXXX/locations/global/workloadIdentityPools/xxxxxxx/providers/xxxxxxx",
    "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
    "token_url": "https://sts.googleapis.com/v1/token",
    "credential_source": {
        "environment_id": "aws1",
        "region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
        "url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
        "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
    },
    "token_info_url": "https://sts.googleapis.com/v1/introspect",
    "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/[email protected]:generateAccessToken"
}

[mattmatters]

Hey @weaversam8 I've actually gotten this to work with my modifications https://github.com/peburrows/goth/pull/184 and https://github.com/peburrows/goth/pull/183.

I figured it would be a bit much with dependencies to add aws directly (and expect goth to know where to pull in my access creds since we use access tokens in dev environments and roles in production) so I'm dynamically building out the credentials by basically doing this

defmodule ExampleApp.Goth do
  def child_spec(opts \\ []) do
    name = Keyword.get(opts, :name, __MODULE__)
    Goth.child_spec(name: name, source: {:mfa, {__MODULE__, :build_credentials, []}})
  end

  def build_credentials() do
    goth_conf = Application.get_env(:example_app, __MODULE__, [])

    workload_identity_pool_provider_name =
      Keyword.get(goth_conf, :workload_identity_pool_provider_name, "replace_me")

    service_account_unique_id = Keyword.get(goth_conf, :service_account_unique_id, "replace_me")
    aws_conf = ExAws.Config.new(:sts)

    url =
      "https://sts.#{aws_conf.region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"

    audience = Path.join("//iam.googleapis.com", workload_identity_pool_provider_name)

    service_account_impersonation_url =
      "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/#{service_account_unique_id}:generateAccessToken"

    {:ok, headers} =
      ExAws.Auth.headers(
        :post,
        url,
        :sts,
        aws_conf,
        [{"x-goog-cloud-target-resource", audience}],
        ""
      )

    subject_token = %{
      "url" => url,
      "method" => "POST",
      "headers" => Enum.map(headers, fn {key, val} -> %{"key" => key, "value" => val} end)
    }

    credentials = %{
      "audience" => audience,
      "service_account_impersonation_url" => service_account_impersonation_url,
      "subject_token_type" => "urn:ietf:params:aws:token-type:aws4_request",
      "token_url" => "https://sts.googleapis.com/v1/token",
      "subject_token" =>
        subject_token
        |> JSON.encode!()
        |> URI.encode(fn char -> char == 47 or URI.char_unreserved?(char) end)
    }

    {:workload_identity, credentials}
  end
end

[weaversam8]

Thanks for the tip @mattmatters! I ended up adding a PR to add AWS support specifically in #186, but your comment helped me track down two issues for my fix! (Both the oddity with URI encoding and the idea to use ExAws to sign the headers came from your comment! ExAws was a huge win because other AWS signature libraries will add the x-amz-content-hash header which breaks GCP Workload Federation.)

For fetching credentials, I focused on the path that's suggested based on GCP credentials.json to pull from the EC2 metadata server. I got this to work locally with aws-vault exec --ec2-server ..., but it would probably be easy to extend what I've written to support the typical environment variables too.

[mattmatters]

Both the oddity with URI encoding and the idea to use ExAws to sign the headers came from your comment!

:heart: So glad to hear it! That uri oddity had me stumped for a while.

[weaversam8]

You know what, funnily enough me too! I just did this implementation for an Erlang project and spent hours fixing the same bug. Wish I had remembered this time 😅

[jamesakers]

Is there a path forward on approval to support AWS Workload Federation?

I extended @weaversam8's feature/aws-workload-federation using a modified credential source (see below just for fun) to support ECS IDMS versions 3 and 4 out of convenience. For my use case, since GCP doesn't export the credentials source I am using, @mattmatters' approach actually seems more appropriate and fits within https://cloud.google.com/iam/docs/workload-identity-federation-with-other-clouds#advanced_scenarios.

{
   "audience": "...",
   "credential_source": {
      "environment_id": "aws_ecs1",
      "url": "http://169.254.170.2{AWS_CONTAINER_CREDENTIALS_RELATIVE_URI}"
      "region_url": "{ECS_CONTAINER_METADATA_URI}/task"
      "regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15",
   },
   "service_account_impersonation_url": "...",
   "subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
   "token_url": "https://sts.googleapis.com/v1/token",
   "type": "external_account",
   "universe_domain": "googleapis.com"
}