operator-framework
operator-sdk run bundle $BUNDLE_IMG on testdata/go/v3/memcached-operator
Bug Report
What did you do?
- run command
operator-sdk run bundle $BUNDLE_IMG --verbose
What did you expect to see?
-
memcached-operatorshould be installed successfully
What did you see instead? Under which circumstances?
- command did not finished successfully
[root@st2-bastion-dev21 memcached-operator]# operator-sdk run bundle $BUNDLE_IMG
INFO[0012] Successfully created registry pod: docker-io-doveyoung-memcached-operator-bundle-v0-0-1
INFO[0012] Created CatalogSource: memcached-operator-catalog
INFO[0012] OperatorGroup "operator-sdk-og" created
INFO[0012] Created Subscription: memcached-operator-v0-0-1-sub
INFO[0024] Approved InstallPlan install-ffjjq for the Subscription: memcached-operator-v0-0-1-sub
INFO[0024] Waiting for ClusterServiceVersion "memcached-operator-system/memcached-operator.v0.0.1" to reach 'Succeeded' phase
INFO[0024] Waiting for ClusterServiceVersion "memcached-operator-system/memcached-operator.v0.0.1" to appear
INFO[0036] Found ClusterServiceVersion "memcached-operator-system/memcached-operator.v0.0.1" phase: Pending
INFO[0038] Found ClusterServiceVersion "memcached-operator-system/memcached-operator.v0.0.1" phase: InstallReady
INFO[0039] Found ClusterServiceVersion "memcached-operator-system/memcached-operator.v0.0.1" phase: Installing
FATA[0120] Failed to run bundle: error waiting for CSV to install: timed out waiting for the condition
- deployment
memcached-operator-controller-managercannot rollout
[root@st2-bastion-dev21 memcached-operator]# oc get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
memcached-operator-controller-manager 0/1 0 0 9m56s
- describe RS
oc describe rs memcached-operator-controller-manager-58759db6bcsaid permission error
Warning FailedCreate 6m2s replicaset-controller Error creating: pods "memcached-operator-controller-manager-58759db6bc-dtlqc" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.seccomp.security.alpha.kubernetes.io/pod: Forbidden: seccomp may not be set pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/kube-rbac-proxy: Forbidden: seccomp may not be set pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/manager: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden: not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork": Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
Warning FailedCreate 25s (x9 over 6m1s) replicaset-controller (combined from similar events): Error creating: pods "memcached-operator-controller-manager-58759db6bc-nbdcw" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.seccomp.security.alpha.kubernetes.io/pod: Forbidden: seccomp may not be set pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/kube-rbac-proxy: Forbidden: seccomp may not be set pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/manager: Forbidden: seccomp may not be set provider "anyuid": Forbidden: not usable by user or serviceaccount provider "nonroot": Forbidden:
not usable by user or serviceaccount provider "hostmount-anyuid": Forbidden: not usable by user or serviceaccount provider "machine-api-termination-handler": Forbidden: not usable by user or serviceaccount provider "hostnetwork":
Forbidden: not usable by user or serviceaccount provider "hostaccess": Forbidden: not usable by user or serviceaccount provider "node-exporter": Forbidden: not usable by user or serviceaccount provider "privileged": Forbidden: not usable by user or serviceaccount]
Environment
Operator type:
go
Kubernetes cluster type:
OpenShift
$ operator-sdk version
$ go version (if language is Go)
operator-sdk version: "v1.8.0-ocp", commit: "bf96b7604eadb82f173df1c60e6999c2d17695fb", kubernetes version: "v1.20.2", go version: "go1.16.6", GOOS: "linux", GOARCH: "amd64"
go version go1.18.2 linux/amd64
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.1", GitCommit:"3ddd0f45aa91e2f30c70734b175631bec5b5825a", GitTreeState:"clean", BuildDate:"2022-05-24T12:26:19Z", GoVersion:"go1.18.2", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5+012e945", GitCommit:"3c28e7a79b58e78b4c1dc1ab7e5f6c6c2d3aedd3", GitTreeState:"clean", BuildDate:"2022-07-13T08:38:41Z", GoVersion:"go1.17.12", Compiler:"gc", Platform:"linux/amd64"}
Possible Solution
Additional context
I can workaround it by run this command
oc adm policy add-scc-to-user privileged -z memcached-operator-controller-manager -n memcached-operator-system
I believe @jmrodri is already looking into some issues related to SCC and the operator-sdk run bundle(-upgrade) commands. I figured mentioning him may help him see this issue. Thanks for filing the issue @dove-young !
So this actually seems to be related to the operand image not running as a non-root user. I think the steps to fix this are:
- [ ] Create a custom memcached image that we maintain for our samples. This should be made to use a non-root user.
- [ ] Update the generated samples to use the custom image.
Okay, so looking into this some more it seems like my last comment was actually incorrect (although it could still be something that needs to be resolved). My bad for not taking a more detailed look at the error messages before posting my last comment :sweat_smile:
This part of the error states the problem is setting the seccomp:
Error creating: pods "memcached-operator-controller-manager-58759db6bc-dtlqc" is forbidden: unable to validate against any security context constraint: [pod.metadata.annotations.seccomp.security.alpha.kubernetes.io/pod: Forbidden: seccomp may not be set pod.metadata.annotations.container.seccomp.security.alpha.kubernetes.io/kube-rbac-proxy
Since all our current samples in the testdata/ folder are set up to set the seccomp I would expect that it would fail when used on a cluster where that isn't allowed to be set. This makes me suspect that the version of OpenShift that you are attempting to run the bundle on doesn't allow the seccomp to be set in the way we are doing in our samples.
@dove-young would you mind sharing what version of OpenShift you are using?
Another thing to mention is that our testdata go samples are generated and during the generation we replace a section that is normally commented as such:
# TODO(user): For common cases that do not require escalating privileges
# it is recommended to ensure that all your Pods/Containers are restrictive.
# More info: https://kubernetes.io/docs/concepts/security/pod-security-standards/#restricted
# Please uncomment the following code if your project does NOT have to work on old Kubernetes
# versions < 1.19 or on vendors versions which do NOT support this field by default (i.e. Openshift < 4.11 ).
# seccompProfile:
# type: RuntimeDefault
As per the comment, if you are using an OpenShift version < 4.11 then you would need to remove the seccompProfile section or use the workaround you provided to get it to work.
@dove-young I just wanted to follow up. If you comment out the seccompProfile sections of the files under the config/ directory for the operator does it work as expected?
Closing. @dove-young if you are still encountering this issue, feel free to reopen.