exclusions based on container name in Assign CRD

[developer-guy]

This issue is based on the conversation in Slack: https://openpolicyagent.slack.com/archives/CDTN970AX/p1628080547007600

Let's assume that I'm a Kubernetes Administrator and I want to enforce some organizational policies across Kubernetes environments. But before enforcing policies I just want to mutate my company's domain team's Pod manifests with the corresponding security configurations by using the OPA GK Mutating feature. In such a case I do not know the container name of the Pod because I'm not responsible for the manifest, I'm just a Kubernetes admin, I also do not know how many containers will get run within the Pod in runtime, I mean by that it might be added some sidecars to the Pod at before running like istio-proxy. So, I just want to mutate my application container, not the sidecars, because this might affect the sidecars and I don't want to do that. I actually want to have control over my Pod while mutating them.

apiVersion: mutations.gatekeeper.sh/v1alpha1
kind: Assign
metadata:
  name: demo-privileged
spec:
  applyTo:
  - groups: [""]
    kinds: ["Pod"]
    versions: ["v1"]
  match:
    scope: Namespaced
    kinds:
    - apiGroups: ["*"]
      kinds: ["Pod"]
    namespaces: ["bar"]
  location: "spec.containers[name:{{.metadata.name}}].securityContext.privileged" ## something like that can
  ## help me to get the container name even if I don't know the container name. metadata specifies the properties defined 
## in the .metadata section of the Pod, Deployment etc.
  parameters:
    assign:
      value: false

I just need to know the container name as a Kubernetes administrator, am I explained it well, if not, please tell me in the comments then I'll try to explain it in more detail 🙏

[developer-guy]

cc: @dentrax @sozercan

[maxsmythe]

My understanding is that you want a way to mutate only the main container for a Pod, but not other containers that may be injected by sidecars later. Your problem is you have no way of identifying that main container. Is that right?

A few questions:

1. It looks like you are suggesting a convention where the main container name == pod name, deployment name, or similar. Is there anything to enforce that convention? If there is something enforcing this convention, would it be simpler to require all developers name primary container main? If there isn't something enforcing a naming convention, how would you ever reliably detect the primary container?
2. What happens if the user wants to supply more than one container, do you want to mutate both of them? How would you do this if you only expect one name?

I could see creating some kind of filtering for the glob notation, so users could maybe do things like:

• Mutate all resources except those with a sidecar- prefix
• Mutate all resources that have a main- prefix
• Mutate all resources except for specific key values (e.g. istio-proxy)

If this were validation, the above would open up the possibility of security holes (e.g. bypass checks by adding a sidecar- prefix), but mutation is mostly about convenience and defaulting, not security.

If we do something like this, it should definitely generalize for all types or resources and keyed lists, not just containers on pods.

@ritazh @sozercan @shomron @willbeason @julianKatz thoughts?

[maxsmythe]

I should also add that there is probably no way to reliably detect "only containers that were part of the original request"

[developer-guy]

My understanding is that you want a way to mutate only the main container for a Pod, but not other containers that may be injected by sidecars later. Your problem is you have no way of identifying that main container. Is that right?

Exactly! Because think of the problem this way, I'm the Kubernetes Administrator, and I'm responsible for maintaining the security across our company's Kubernetes clusters. I want to enforce that our Pods should be running with the most common&important security fields are filled before deploying them. I thought I could use the mutating feature of the OPA GK for that, but here is the limitation that I'm trying to explain, I do not know about my domain team's Pods and the containers within them. So, IMHO, there are two ways of doing this. The first one is domain teams should apply Assign CR with the corresponding container name, and this means that there are lots of Assign CR used in my cluster (I'm not sure about whether it is ok or not btw). The second one is that we should enforce some standards to determine which one is the application container or at least what is the name of the application container using something like I mentioned above, making Pod name and the container name equivalent so that we can use {{.metadata.name}} template for our Assign CR.

[maxsmythe]

Understood, there are some deeper issues around things like multi-container pods, what conventions are reasonable, and what other linguistic primitives we might add to make this work.

See my previous post for some of the questions I have and some approaches I think may be helpful.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[maxsmythe]

This is still a going concern

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.