Get "Unable to load the proper Managed Identity" error when switching from IDownstreamApi#CallApiForAppAsync to IDownstreamApi#CallApiForUserAsync

[cabillin]

Microsoft.Identity.Web Library

Microsoft.Identity.Web

Microsoft.Identity.Web version

3.6.0

Web app

Sign-in users and call web APIs

Web API

Not Applicable

Token cache serialization

In-memory caches

Description

I have a GraphQL service (ASP.NET Core, HotChocolate) that can successfully call into our RESTful API service when I use IDownstreamApi#CallApiForAppAsync. However, one of the APIs we want to call uses the user credentials to access a dependency, so I need to switch to IDownstreamApi#CallApiForUserAsync. Once I make that change, I get the following error: The certificate could not be loaded from the ClientCredentials: ManagedIdentityCredential authentication failed: [Managed Identity] Error Message: Unable to load the proper Managed Identity.

IDownstreamApi#CallApiForUserAsync works in localhost (when it falls back to client credentials in KeyVault).

After switching to no longer using the IDownstreamApi helper class (i.e. option 3-ish directly using CreateAuthorizationHeaderAsync with RequestAppToken = false when I need the onBehalfOf flow), I no longer had the problem. I think this might be related to https://github.com/AzureAD/microsoft-identity-web/issues/2912.

Reproduction steps

See the code snippets section.

Error message

The certificate could not be loaded from the ClientCredentials: ManagedIdentityCredential authentication failed: [Managed Identity] Error Message: Unable to load the proper Managed Identity. Managed Identity Correlation ID: 6258c513-3603-4ad8-a453-daab31e0dc48 Use this Correlation ID for further investigation.\nSee the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/managedidentitycredential/troubleshoot", "stackTrace": " at Microsoft.Identity.ServiceEssentials.MiseDefaultAuthorizationHeaderProvider.GetCertificateAsync(CredentialDescription certificateDescription)\r\n at Microsoft.Identity.ServiceEssentials.MiseDefaultAuthorizationHeaderProvider.GetCertificateFromClientCredentialsAsync(IEnumerable1 clientCredentials)\r\n at Microsoft.Identity.ServiceEssentials.MiseDefaultAuthorizationHeaderProvider.CreateS2SOutboundPolicyAsync(IEnumerable1 scopes, ValueTuple2 schemeAndTokenTypeTuple)\r\n at Microsoft.Identity.ServiceEssentials.MiseDefaultAuthorizationHeaderProvider.CreateAuthorizationHeaderUsingOutboundPolicyAsync(IEnumerable1 scopes, AuthorizationHeaderProviderOptions authorizationHeaderProviderOptions, ClaimsPrincipal claimsPrincipal, CancellationToken cancellationToken)\r\n at Microsoft.Identity.ServiceEssentials.MiseDefaultAuthorizationHeaderProvider.CreateAuthorizationHeaderAsync(IEnumerable`1 scopes, AuthorizationHeaderProviderOptions authorizationHeaderProviderOptions, ClaimsPrincipal claimsPrincipal, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Web.DownstreamApi.UpdateRequestAsync(HttpRequestMessage httpRequestMessage, HttpContent content, DownstreamApiOptions effectiveOptions, Boolean appToken, ClaimsPrincipal user, CancellationToken cancellationToken)\r\n at Microsoft.Identity.Web.DownstreamApi.CallApiInternalAsync(String serviceName, DownstreamApiOptions effectiveOptions, Boolean appToken, HttpContent content, ClaimsPrincipal user, CancellationToken cancellationToken)\r\n at {my code}

Id Web logs

No response

Relevant code snippets

My appSettings file looks like the following:

{
  "AllowedHosts": "*",
  "AzureMonitor": {
    "ConnectionString": "redacted"
  },
  "AzureAd": {
    "Instance": "https://login.microsoftonline.com/",
    "TenantId": "redacted",
    "ClientId": "PPE app registration",
    "ClientCredentials": [
      {
        "SourceType": "SignedAssertionFromManagedIdentity",
        "ManagedIdentityClientId": "User assigned MI in PPE that is registered as a federated credential with the PPE app registration",
        "TokenExchangeUrl": "api://AzureADTokenExchange"
      },
      {
        "SourceType": "KeyVault",
        "KeyVaultUrl": "redacted",
        "KeyVaultCertificateName": "redacted"
      }
    ],
    "SendX5c": true
  },
  "DownstreamApis": {
    "RESTful_APIs_service_name": {
      "BaseUrl": "redacted",
      "Scopes": [ "redacted/.default" ]
    },
  },
  "Logging": {
    "LogLevel": {
      "Default": "Trace",
      "Microsoft.AspNetCore": "Warning",
      "Microsoft.Identity": "Warning"
    }
  }
}

The GraphQL Web App has EasyAuth turned on.

My code looks like the following:

public class Program
{
    public static void Main(string[] args)
    {
        var builder = WebApplication.CreateBuilder(args);
        var startup = new Startup(builder.Environment);
        startup.ConfigureServices(builder);

        var app = builder.Build();
        startup.Configure(app, app.Environment);
        app.Run();
    }
}

public class Startup
{
...
        public void ConfigureServices(WebApplicationBuilder builder)
        {
            services.AddSingleton<IConfigurationRoot>(builder.Configuration);

            services.AddAuthentication(S2SAuthenticationDefaults.AuthenticationScheme)
    .AddMiseWithDefaultModules(builder.Configuration);

            services.AddCors(...);

            ...
            var openTelBuilder = services
            .AddOpenTelemetry()
            ...

            services.Configure<ForwardedHeadersOptions>(...);

            services.AddMemoryCache();

            ...

            var tokenAcquirerFactory = TokenAcquirerFactory.GetDefaultInstance("AzureAD");
tokenAcquirerFactory.Services.AddDownstreamApis(tokenAcquirerFactory.Configuration.GetSection("DownstreamApis"));
            services.AddSingleton<IServiceProvider>(tokenAcquirerFactory.Build());

            var hotChocolateBuilder = services
            ...
            .AddGraphQLServer()
            ...
        }

        public void Configure(WebApplication app, IHostEnvironment env)
        {
            app.UseDeveloperExceptionPage()
                .UseExceptionHandler(...)
                .UseHttpsRedirection()
                .UseRouting()
                .UseForwardedHeaders(...)
                .UseCors();

            app.MapGet("/", () => $"You found {redacted} GraphQL! EnvironmentName={env.EnvironmentName}");
            app.UseEndpoints(endpoints =>
            {
                endpoints.MapGraphQLHttp("/redacted").WithOptions(...).RequireCors();
                endpoints.MapGraphQLSchema("/redacted");
                endpoints.MapNitroApp("/redacted").WithOptions(...);
            });
        }

}

public class AppService
{
    ...

    public async Task<TOutput> MakeTheApiCall(ClaimsPrincipal user, ...)
    {
        ...
            var api = serviceProvider.GetRequiredService<IDownstreamApi>();
            var result = await api.CallApiForUserAsync(serviceName, o => { o.HttpMethod = HttpMethod.Post.ToString(); o.RelativePath = relativeUri; }, user: user, content: new StringContent(serializedBody, Encoding.UTF8, "application/json"), cancellationToken: cancellationToken);
            ...
            data = Deserializer(result.Content, deserializationCallback);
        ...
        return data;
    }

public class Query
{
    ...

    [UsePaging]
    [GraphQLDescription("Redacted.")]
    public async Task<TOutput> QueryStuff(CancellationToken cancellationToken, IResolverContext context, [Service] IAppService appService, ...)
    {
        var user = context.GetUser();
        if (!user.Identity.IsAuthenticated)
        {
            throw new Exception("Requires user to be authenticated");
        }
        TOutput stuff = await appService.QueryCdFeatures(user, ...);
        ...
        return stuff;
    }
}

Regression

No response

Expected behavior

IDownstreamApi#CallApiForUserAsync works with SignedAssertionFromManagedIdentity