azure-cli icon indicating copy to clipboard operation
azure-cli copied to clipboard

Unable to fetch the AKS config

Open codingCoffee opened this issue 1 year ago • 1 comments

Describe the bug

Running the following command

az aks get-credentials --resource-group XXX --name XXX --overwrite-existing

returns the error

User '[email protected]' does not exist in MSAL token cache. Run `az login`.

Even though I'm already logged in. Here's the output of az account list

[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "isDefault": true,
    "managedByTenants": [],
    "name": "XXXXXXXXX XXXXX XXXXXXXXXXX",
    "state": "Enabled",
    "tenantDefaultDomain": "xxxxx.xxx",
    "tenantDisplayName": "Grapevine",
    "tenantId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
    "user": {
      "name": "[email protected]",
      "type": "user"
    }
  }
]

Related command

az aks get-credentials

Errors

User '[email protected]' does not exist in MSAL token cache. Run `az login`.

Issue script & Debug output

az aks get-credentials --debug --resource-group xx-xxxxxx-xxxx-xxxx-xxx --name xxx-xxxxxx-xxxxxxx-xxxxx-xxxxxxxxxxxx-xx --overwrite-existing
cli.knack.cli: Command arguments: ['aks', 'get-credentials', '--debug', '--resource-group', 'xx-xxxxxx-xxxx-xxxx-xxx', '--name', 'xxx-xxxxxx-xxxxxxx-xxxxx-xxxxxxxxxxxx-xx', '--overwrite-existing']
cli.knack.cli: __init__ debug log:
Enable color in terminal.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7ff74ff32340>, <function OutputProducer.on_global_arguments at 0x7ff74fe66a20>, <function CLIQuery.on_global_arguments at 0x7ff74fcb8360>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Command index version or cloud profile is invalid or doesn't match the current command.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core: Command index has been invalidated.
cli.azure.cli.core: No module found from index for '['aks', 'get-credentials', '--debug', '--resource-group', 'xx-xxxxxx-xxxx-xxxx-xxx', '--name', 'xxx-xxxxxx-xxxxxxx-xxxxx-xxxxxxxxxxxx-xx', '--overwrite-existing']'
cli.azure.cli.core: Loading all modules and extensions
cli.azure.cli.core: Discovered command modules: ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'compute_recommender', 'config', 'configure', 'consumption', 'container', 'containerapp', 'cosmosdb', 'databoxedge', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'identity', 'interactive', 'iot', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'marketplaceordering', 'monitor', 'mysql', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'resource', 'role', 'search', 'security', 'servicebus', 'serviceconnector', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'synapse', 'util', 'vm']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name                  Load Time    Groups  Commands
cli.azure.cli.core: acr                       0.220        36       149
cli.azure.cli.core: acs                       0.026        14        76
cli.azure.cli.core: advisor                   0.001         3         6
cli.azure.cli.core: ams                       0.005        22       100
cli.azure.cli.core: apim                      0.006        14        69
cli.azure.cli.core: appconfig                 0.003         9        47
cli.azure.cli.core: appservice                0.058        79       270
cli.azure.cli.core: aro                       0.007         1        10
cli.azure.cli.core: backup                    0.003        16        60
cli.azure.cli.core: batch                     0.026        34       102
cli.azure.cli.core: batchai                   0.002        10        30
cli.azure.cli.core: billing                   0.009        19        53
cli.azure.cli.core: botservice                0.005        12        42
cli.azure.cli.core: cdn                       0.210         8        50
cli.azure.cli.core: cloud                     0.001         1         7
cli.azure.cli.core: cognitiveservices         0.002        10        33
cli.azure.cli.core: compute_recommender       0.002         1         1
cli.azure.cli.core: config                    0.001         2         7
cli.azure.cli.core: configure                 0.001         2         5
cli.azure.cli.core: consumption               0.013         8         9
cli.azure.cli.core: container                 0.006         1        11
cli.azure.cli.core: containerapp              0.137        37       123
cli.azure.cli.core: cosmosdb                  0.013        58       199
cli.azure.cli.core: databoxedge               0.005         5        28
cli.azure.cli.core: dla                       0.004        23        62
cli.azure.cli.core: dls                       0.002         7        41
cli.azure.cli.core: dms                       0.003         3        22
cli.azure.cli.core: eventgrid                 0.004        25        96
cli.azure.cli.core: eventhubs                 0.010        13        19
cli.azure.cli.core: extension                 0.001         1         7
cli.azure.cli.core: feedback                  0.001         1         2
cli.azure.cli.core: find                      0.001         1         1
cli.azure.cli.core: hdinsight                 0.005         8        39
cli.azure.cli.core: identity                  0.001         2        11
cli.azure.cli.core: interactive               0.000         1         1
cli.azure.cli.core: iot                       0.071        19        82
cli.azure.cli.core: keyvault                  0.007        20       113
cli.azure.cli.core: kusto                     0.002         3        14
cli.azure.cli.core: lab                       0.002        11        34
cli.azure.cli.core: managedservices           0.001         3         8
cli.azure.cli.core: maps                      0.001         5        13
cli.azure.cli.core: marketplaceordering       0.002         1         2
cli.azure.cli.core: monitor                   0.396        18        61
cli.azure.cli.core: mysql                     0.091        15        53
cli.azure.cli.core: netappfiles               0.030         8        17
cli.azure.cli.core: network                   0.181       103       338
cli.azure.cli.core: policyinsights            0.010         9        17
cli.azure.cli.core: privatedns                0.018        14        60
cli.azure.cli.core: profile                   0.001         2         8
cli.azure.cli.core: rdbms                     0.018        49       202
cli.azure.cli.core: redis                     0.002         7        38
cli.azure.cli.core: relay                     0.019         7         8
cli.azure.cli.core: resource                  0.011        51       231
cli.azure.cli.core: role                      0.002        17        61
cli.azure.cli.core: search                    0.006         7        19
cli.azure.cli.core: security                  0.010        48        98
cli.azure.cli.core: servicebus                0.006        12        15
cli.azure.cli.core: serviceconnector          0.020        20       309
cli.azure.cli.core: servicefabric             0.011        27        80
cli.azure.cli.core: signalr                   0.003         9        34
cli.azure.cli.core: sql                       0.013        56       215
cli.azure.cli.core: sqlvm                     0.022         4        20
cli.azure.cli.core: storage                   0.046        59       273
cli.azure.cli.core: synapse                   0.011        54       246
cli.azure.cli.core: util                      0.001         3         7
cli.azure.cli.core: vm                        0.046        58       233
cli.azure.cli.core: Total (66)                1.851      1206      4697
cli.azure.cli.core: Loaded 1192 groups, 4697 commands.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core: Updated command index in 0.003 seconds.
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7ff74eb29580>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to '/home/xxxx/.azure/commands/2024-08-28.14-53-26.aks_get-credentials.335378.log'.
az_command_data_logger: command args: aks get-credentials --debug --resource-group {} --name {} --overwrite-existing
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x7ff74eb7b380>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x7ff74eb91580>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x7ff74eb916c0>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7ff74cbe4680>]
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7ff74fe66ac0>, <function CLIQuery.handle_query_parameter at 0x7ff74fcb8400>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x7ff74eb91620>]
cli.azure.cli.core.commands.client_factory: Getting management service client client_type=ContainerServiceClient
cli.azure.cli.core.auth.persistence: build_persistence: location='/home/xxxx/.azure/msal_token_cache.json', encrypt=False
cli.azure.cli.core.auth.binary_cache: load: /home/xxxx/.azure/msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: Initializing with Entra authority: https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142
msal.authority: openid_config("https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/v2.0/.well-known/openid-configuration") = {'token_endpoint': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/900cd913-e6d9-401f-92ce-70527cdf8142/kerberos', 'tenant_region_scope': 'AS', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? None
msal.application: get_accounts(username='[email protected]') finds no account. If tokens were acquired without 'profile' scope, they would contain no username for filtering. Consider calling get_accounts(username=None) instead.
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/nix/store/ikls9n31jvixjq9p7sn1zr2jasg2z7n8-python3.12-knack-0.12.0/lib/python3.12/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 701, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/__init__.py", line 334, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/command_operation.py", line 112, in handler
    client = self.client_factory(self.cli_ctx, command_args) if self.client_factory else None
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/lib/python3.12/site-packages/azure/cli/command_modules/acs/_client_factory.py", line 27, in cf_managed_clusters
    return get_container_service_client(cli_ctx).managed_clusters
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/lib/python3.12/site-packages/azure/cli/command_modules/acs/_client_factory.py", line 19, in get_container_service_client
    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_CONTAINERSERVICE, subscription_id=subscription_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/client_factory.py", line 83, in get_mgmt_service_client
    client, _ = _get_mgmt_service_client(cli_ctx, client_type, subscription_id=subscription_id,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/commands/client_factory.py", line 236, in _get_mgmt_service_client
    credential, subscription_id, _ = profile.get_login_credentials(
                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/_profile.py", line 366, in get_login_credentials
    credential = self._create_credential(account, client_id=client_id)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/_profile.py", line 624, in _create_credential
    return identity.get_user_credential(username_or_sp_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/auth/identity.py", line 245, in get_user_credential
    return UserCredential(self.client_id, username, **self._msal_public_app_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/jn55sh7c2alakzfii9n19wag9jc1fv0n-python3.12-azure-cli-core-2.63.0/lib/python3.12/site-packages/azure/cli/core/auth/msal_authentication.py", line 58, in __init__
    raise CLIError("User '{}' does not exist in MSAL token cache. Run `az login`.".format(username))
knack.util.CLIError: User '[email protected]' does not exist in MSAL token cache. Run `az login`.

cli.azure.cli.core.azclierror: User '[email protected]' does not exist in MSAL token cache. Run `az login`.
az_command_data_logger: User '[email protected]' does not exist in MSAL token cache. Run `az login`.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7ff74eb29800>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 2.489 seconds (init: 0.371, invoke: 2.119)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3873 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12 /nix/store/5vmz4nvsdgv5359pk6k6qwis2z6zli05-python3.12-azure-cli-telemetry-1.1.0/lib/python3.12/site-packages/azure/cli/telemetry/__init__.py /home/xxxx/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.

Expected behavior

I expect my kubeconfig to be populated

Environment Summary

azure-cli                         2.63.0

core                              2.63.0
telemetry                          1.1.0

Dependencies:
msal                              1.29.0
azure-mgmt-resource               23.1.1

Python location '/nix/store/04gg5w1s662l329a8kh9xcwyp0k64v5a-python3-3.12.4/bin/python3.12'
Extensions directory '/home/xxxx/.azure/cliextensions'

Python (Linux) 3.12.4 (main, Jun  6 2024, 18:26:44) [GCC 13.3.0]

Legal docs and information: aka.ms/AzureCliLegal


Your CLI is up-to-date.

Additional context

No response

codingCoffee avatar Aug 28 '24 09:08 codingCoffee

route to service team

yonzhan avatar Aug 28 '24 11:08 yonzhan

@codingCoffee how you found a workaround for this?

I'm seeing something similar, with these log entries:

cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.
cli.azure.cli.core._session: Skipping update of file /nix/store/6ldp9ls6pc4ib8ij86nrqwhrplhx8fqf-python3.12-azure-cli-2.63.0/etc/azure/commandIndex.json due to immutable directory.

I've tried running using buildFHSUserEnv, but leads to same result.{

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
  };

  outputs = { self, nixpkgs }:
    let
      system = "x86_64-linux";
      pkgs = nixpkgs.legacyPackages.${system};
      fhs = pkgs.buildFHSUserEnv {
        name = "fhs-shell";
        targetPkgs = pkgs: [
          (pkgs.azure-cli.withExtensions [ pkgs.azure-cli.extensions.account ])
          pkgs.azure-functions-core-tools
        ];
      };
    in
      {
        devShells.${system}.default = fhs.env;
      };
}

morten-b avatar Nov 08 '24 07:11 morten-b

I used docker to get around the issue. Install azure cli in docker, link ~/.azure volume on host to the /root/.azure volume inside the container. Use this for the initial setup, then you should be able to use azure on the host

codingCoffee avatar Nov 08 '24 08:11 codingCoffee

I used docker to get around the issue. Install azure cli in docker, link ~/.azure volume on host to the /root/.azure volume inside the container. Use this for the initial setup, then you should be able to use azure on the host

Ah great. How about exposing the az bin to the host?

morten-b avatar Nov 08 '24 09:11 morten-b

Ah great. How about exposing the az bin to the host?

Yea you could do that as well, but after the initial setup, there's no need for this. I think you could also use something like asdf to install multiple versions of azure-cli, if you really don't want to use nix's azure-cli package

codingCoffee avatar Nov 09 '24 07:11 codingCoffee

Hi @codingCoffee, glad you found a workaround with docker. Is the original issue still occurring for you?

mbifeld avatar Sep 08 '25 20:09 mbifeld