Exemptions in isolated mode

[rhtenhove]

Is your feature request related to a problem?

Wanting to deploy a manifest which violates the vCluster isolated mode admission requirements. This can be anything, for example mounting a certain directory on the host.

This is possible with PodSecurity (see suggested solution). Sadly configuring this for the vCluster kube-apiserver doesn't have the desired effect as vCluster isolated mode handles this differently.

Which solution do you suggest?

PodSecurity allows to specify users (and more) in an AdmissionConfiguration manifest for the kube-apiserver, which then allows manifests to be deployed as that user which will then not be checked for any violations.

For example:

apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
  - name: PodSecurity
    configuration:
      apiVersion: pod-security.admission.config.k8s.io/v1beta1
      kind: PodSecurityConfiguration
      exemptions:
        usernames:
          - benchperson

Have this work for the vCluster kube-apiserver too (or use a similar implementation working for the vCluster syncer).

Which alternative solutions exist?

None as far as I know. Unable to have violating manifests in an isolated vcluster.

Additional context

As discussed with Fabian Kramm and Oleg Matskiv on Slack

[FabianKramm]

@rhtenhove thanks for creating this issue! I like the idea, but I believe implementation will be difficult, because vcluster is doing the validation on syncer level, while k8s is doing it as an admission controller during api validation. The problem with that is that we would need to track who created which object, which is a pretty difficult task as most pods are created by deployments and even those would need an annotation or another logic to determine this.

I'm not sure if we should actually implement this as you could disable the vcluster specific pod security standard enforcement through isolation.podSecurityStandard: '' and actually use a v1.24 vcluster and in there the regular kubernetes admission controller that is configurable like this. This would probably already solve your problem as the v1.24 version also works on older Kubernetes clusters.