V2 Kubeconfig is missing "profile" argument

[shanestrasser]

I created a kubeconfig using oci-cli tool version 2.6.12 (OS is Windows 10).

oci ce cluster create-kubeconfig --cluster-id $CLUSTER --file $HOME\.kube_oci\config_2 --region us-ashburn-1 --profile PROFILE_2 --token-version 2.0.0

The resulting exec snippet of the kubeconfig is

      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - ce
      - cluster
      - generate-token
      - --cluster-id
      - ocid1.cluster.oc1.abcdefg
      - --region
      - us-ashburn-1
      command: oci
      env: []

Trying to run kubectl version results in the following:

E1119 10:46:27.645158    1976 round_trippers.go:174] CancelRequest not implemented by *exec.roundTripper
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.6", GitCommit:"96fac5cd13a5dc064f7d9f4f23030a6aeface6cc", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:49Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"windows/amd64"}
error: You must be logged in to the server (the server has asked for the client to provide credentials)

I ended up having to modify the kubeconfig myself to include the profile argument:

      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - ce
      - cluster
      - generate-token
      - --cluster-id
      - ocid1.cluster.oc1.abcdefg
      - --region
      - us-ashburn-1
      - --profile
      - PROFILE_2
      command: oci
      env: []

With those changes, I am able to successfully connect to my cluster.

I'd expect that the generated kubeconfig file didn't require me having to edit it and that the oci-cli create-kubeconfig should be able to pick up the additional arguments and pass them into the generated kubeconfig file.

[paul-hummel-oracle]

@shanestrasser thank you for reporting this issue. We will investigate this further and reply back here with more details.

[kennyunique]

@shanestrasser , your modification to the kubeconfig file is the work-around we recommend to customers. Basically, the Kubeconfig contains an oci-cli generate-token command. By default, it will use the default profile in your oci config to create an auth token. If you try to use that token to authenticate with a cluster that profile has no access to, you will see "You must be logged in to the server" error. The kubeconfig is generated by the API server and the API server would not know what oci profile is used. So, it is not possible to generate a kubeconfig with correct --profile parameters for the embedded "generate-token" command.

[shanestrasser]

Thanks for getting back to me @kennyunique . Having the API server generate the kubeconfig does complicate things... Thinking out loud: when the oci-cli gets the token generated from the API server, could a modification to the file be done by local host to add those parameters before it's written to file? Do we have it documented which fields aren't copied from the oci ce cluster create-kubeconfig command into the Kubeconfig file?

[robertpatrick]

@kennyunique While the workaround works, it is clunky at best. We really should fix this...

[robertpatrick]

@vish1 From looking at the source, fixing this without modifying the API server seems pretty straightforward. Basically, you need to augment the call to create_kubeconfig() in containerengine_cli_extended.py to pass the profile argument value, if any.

When the call from the API server returns and the profile argument is not None, you just decode() and yaml.safe_load() the response, find the user[0].user.exec.args list and simple add --profile and the value of the profile argument to the end of the list before doing anything else. It might require some refactoring of the existing method (and the helper methods that process the merge) to handle the resulting Python dictionary but this seems pretty straightforward to me. What am I missing?

[ajhindle]

I'm using oci cli v3.22.1 and this issue is still there.

@paul-hummel-oracle @kennyunique can we please have an update? @robertpatrick says the fix is pretty straightforward

My config doesn't work unless I put in the --auth security_token reference (see below)

- name: user-<hidden>
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - ce
      - cluster
      - generate-token
      - --cluster-id
      - ocid1.cluster.oc1.<hidden>
      - --region
      - <hidden>
      - --profile
      - ajhindle
      - --auth
      - security_token
      command: oci
      env: []
 

[ararhe]

I'm using oci cli v3.25.0 and this issue is still there. adding --profile to the kubeconfig works fine, but anytime we need to re-run "OCI Ce" to switch between multiple clusters it's overwriting the changes we made in the kubeconfig file https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/clitoken.htm

Interesting oci ce cluster create-kubeconfig getting an argument to generate the kubeconfig but the generated kubeconfig file doesn't have the same argument, this issue is not fixed looks like after 4 years.

If the goal is to generate the file and we need to pass the arguments but we can't use them what is the point of having the profile as an argument and it's always going to use the default profile

https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.30.2/oci_cli_docs/cmdref/ce/cluster/create-kubeconfig.html

[robertpatrick]

@ararhe While I agree with your sentiment, the --profile argument to the OCI CLI command allows the CLI to use the right credentials to talk to the OCI APIs (e.g., the credentials associated with the tenancy where the OKE cluster resides). The argument from the development team that the server doesn't know about the profile so they cannot put it into the KUBECONFIG file is weak, since it can easily be added to what the server returns prior to writing the file.

[ysalitrynskyi]

I'm using oci cli v3.22.1 and this issue is still there.

@paul-hummel-oracle @kennyunique can we please have an update? @robertpatrick says the fix is pretty straightforward

My config doesn't work unless I put in the --auth security_token reference (see below)

- name: user-<hidden>
  user:
    exec:
      apiVersion: client.authentication.k8s.io/v1beta1
      args:
      - ce
      - cluster
      - generate-token
      - --cluster-id
      - ocid1.cluster.oc1.<hidden>
      - --region
      - <hidden>
      - --profile
      - ajhindle
      - --auth
      - security_token
      command: oci
      env: []
 

export OCI_CLI_AUTH=security token
export OCI_CLI_PROFILE=DEFAULT

[robertpatrick]

@ysalitrynskyi it seems clear that none of the contributors think this issue is important. This is why I haven’t wasted my time to fix it myself and submit a PR.