Otel collector RBAC

[iliuta]

Description: When deploying add-ons helm chart with otel collector only activated we see rbac issues in the logs of otel collector.

Repro steps: Activate otel collector and deactivate everything else.

My value file looks like this:

gateway-addons-helm:
  grafana:
    enabled: false
  prometheus:
    enabled: false
  fluent-bit:
    enabled: false
  loki:
    enabled: false
  alloy:
    enabled: false
  tempo:
    enabled: false

  opentelemetry-collector:
    enabled: true
    serviceAccount:
      create: true
      name: otel-collector

Deploy the chart.

The otel collector logs start showing these:

opentelemetry-collector W1212 07:24:04.173005       1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1.Pod: pods is forbidden: User "system:service
account:gateway-api-resources:otel-collector" cannot list resource "pods" in API group "" at the cluster scope
opentelemetry-collector E1212 07:24:04.173074       1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1.Pod: failed to list
*v1.Pod: pods is forbidden: User \"system:serviceaccount:gateway-api-resources:otel-collector\" cannot list resource \"pods\" in API group \"\" at the cluster scope" logger="UnhandledError"
opentelemetry-collector W1212 07:24:06.232637       1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1.ReplicaSet: replicasets.apps is forbidden: U
ser "system:serviceaccount:gateway-api-resources:otel-collector" cannot list resource "replicasets" in API group "apps" at the cluster scope
opentelemetry-collector E1212 07:24:06.232705       1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1.ReplicaSet: failed t
o list *v1.ReplicaSet: replicasets.apps is forbidden: User \"system:serviceaccount:gateway-api-resources:otel-collector\" cannot list resource \"replicasets\" in API group \"apps\" at the cl
uster scope" logger="UnhandledError"
opentelemetry-collector W1212 07:24:07.472463       1 reflector.go:569] k8s.io/[email protected]/tools/cache/reflector.go:251: failed to list *v1.Namespace: namespaces "kube-system" is forbi
dden: User "system:serviceaccount:gateway-api-resources:otel-collector" cannot list resource "namespaces" in API group "" at the cluster scope
opentelemetry-collector E1212 07:24:07.472534       1 reflector.go:166] "Unhandled Error" err="k8s.io/[email protected]/tools/cache/reflector.go:251: Failed to watch *v1.Namespace: failed to
 list *v1.Namespace: namespaces \"kube-system\" is forbidden: User \"system:serviceaccount:gateway-api-resources:otel-collector\" cannot list resource \"namespaces\" in API group \"\" at the
 cluster scope" logger="UnhandledError"

When running a helm template, we can see that the cluster roles and cluster role bindings are not generated.

I've played a little bit with the values and it seems that the rbac manifests are only generated when activating alloy.

gateway-addons-helm:
  grafana:
    enabled: false
  prometheus:
    enabled: false
  fluent-bit:
    enabled: false
  loki:
    enabled: false
  alloy:
    enabled: true
  tempo:
    enabled: false

  opentelemetry-collector:
    enabled: true

Note: If there are privacy concerns, sanitize the data prior to sharing.

Environment:

gateway helm charts versions 1.6.1

Logs:

Include the access logs and the Envoy logs.