helmchart icon indicating copy to clipboard operation
helmchart copied to clipboard

avoid child open port and fix liveness probe on public worker nodes

Open alexfouche opened this issue 2 years ago • 1 comments

On a k8s cluster with public worker node, the child daemonset will be available publicly on port 19999, since it runs with hostNetwork: true

From the Securing Netdata pages, and since there is a Netdata parent accessible from a ClusterIP service, it is possible to avoid that by setting :

child:
  configs:
    netdata:
      data: |
        [web]
          mode = none
(...)

Unfortunately this breaks the liveness and readiness probes.

So i am using

child:
  configs:
    netdata:
      data: |
        [web]
          bind to = localhost:19999
(...)

The issue i have is that the Helm values.yaml does not allow to set the httpGet.host: field, so that it can be set to 127.0.0.1

I tested by editing the daemonset livenessProbe and readinessProbe manually, and it works:

livenessProbe:
      httpGet:
        host: 127.0.0.1
        path: /api/v1/info
        port: http
        scheme: HTTP
(...)

Shouldn't it even be set by default for the Netdata child to not listen on all interfaces, since there will be the Netdata parent it streams to anyway ?

alexfouche avatar Jan 16 '24 15:01 alexfouche

Unfortunately this breaks the liveness and readiness probes.

You can use livenessProbe exec - netdatacli ping. We use it for Docker health checks with disabled web server.

ilyam8 avatar Jan 30 '24 08:01 ilyam8

@ilyam8 That's one solution but unfortunately there is no option to override the probes for the child processes in the helm chart, see #433 which at least adds the host option to it.

BirknerAlex avatar Aug 07 '24 10:08 BirknerAlex

@BirknerAlex I think we need to implement it because it will allow to use web. mode = none. #433 fixes only web bind to = localhost:19999

ilyam8 avatar Aug 07 '24 10:08 ilyam8

Yap, thats the reason why I wrote "partically" in the PR. I don't wanted to touch the helm chart that much (due lack of time) but at least provide one possible solution for that kind of issue (public IP nodes).

BirknerAlex avatar Aug 07 '24 10:08 BirknerAlex