Feature Request: Add annotation for fleet-controller and gitjob deployement.

[amolkharche13]

Is your feature request related to a problem? Yes

Both the fleet-controller and gitjob pods have metrics available that can potentially be sent to our SUSE Observability environment, and this would benefit all Rancher customers that use Fleet and SUSE Observability.

When we are trying to configure through terraform we dont see option to add annotation.

Solution you'd like

The following annotation we need to add in deployment spec:

...
metadata:
  annotations:
        ad.stackstate.com/fleet-controller.check_names: '["openmetrics"]'
        ad.stackstate.com/fleet-controller.init_configs: '[{}]'
        ad.stackstate.com/fleet-controller.instances: |
          [ 
            {
              "prometheus_url": "http://%%host%%:8080/metrics",
              "namespace": "cattle_fleet_system", 
              "metrics": ["*"]
            }
          ]
      creationTimestamp: null
...
# This already exists in the pod spec, the container name needs to match the container that is exposing the openmetrics endpoint
spec:
  containers:
   - name: fleet-controller
...

 ...
metadata:
  annotations:
        ad.stackstate.com/ gitjob.check_names: '["openmetrics"]'
        ad.stackstate.com/ gitjob .init_configs: '[{}]'
        ad.stackstate.com/ gitjob .instances: |
          [ 
            {
              "prometheus_url": "http://%%host%%:8081/metrics",
              "namespace": "cattle_fleet_system", 
              "metrics": ["*"]
            }
          ]
      creationTimestamp: null
...
# This already exists in the pod spec, the container name needs to match the container that is exposing the openmetrics endpoint
spec:
  containers:
   - name:  gitjob  

Alternatives you've considered

It would be nice to add in helm chart flag that will add the stackstate annotations automatically. Maybe something like "enable_observability_metrics: true"

Anything else?

No response

[manno]

We can add extraLabels and extraAnnotations to the helm chart.

[manno]

@amolkharche13 we just published a 0.13 release candidate, which can be configured with these annotations.

[dnoland1]

@amolkharche13 can you check if https://github.com/rancher/fleet/pull/3531 looks correct? I believe the stackstate annotations need to ultimately land on the pod spec and not directly on the deployment spec. Can you try out the RC to confirm it works?

[amolkharche13]

@dnoland1 Yes, the annotations need to be at the pod spec of that deployment.

[manno]

https://fleet.rancher.io/ref-configuration#configure-fleet-install-options-in-rancher describes how to modify Fleet install options in Rancher. With the new RC, the chart supports extraAnnotations, note that the content of the fleet key is a yaml string:

kind: ConfigMap
apiVersion: v1
metadata:
  name: rancher-config
  namespace: cattle-system
data:
  fleet: |
    ad.stackstate.com/fleet-controller.check_names: '["openmetrics"]'
    ad.stackstate.com/fleet-controller.init_configs: '[{}]'
    ad.stackstate.com/fleet-controller.instances: |
      [ 
        {
          "prometheus_url": "http://%%host%%:8080/metrics",
          "namespace": "cattle_fleet_system", 
          "metrics": ["*"]
        }
     ]

[amolkharche13]

As discussed with Patrick, the current configuration only supports the fleet-controller container and does not allow configuration for other containers like gitjob with a different port. Could you please assist us in extending the support to include the gitjob container as well?

...
metadata:
  annotations:
        ad.stackstate.com/fleet-controller.check_names: '["openmetrics"]'
        ad.stackstate.com/fleet-controller.init_configs: '[{}]'
        ad.stackstate.com/fleet-controller.instances: |
          [ 
            {
              "prometheus_url": "http://%%host%%:8080/metrics",
              "namespace": "cattle_fleet_system", 
              "metrics": ["*"]
            }
          ]
      creationTimestamp: null
...
# This already exists in the pod spec, the container name needs to match the container that is exposing the openmetrics endpoint
spec:
  containers:
   - name: fleet-controller
...

Gitjob

 ...
metadata:
  annotations:
        ad.stackstate.com/ gitjob.check_names: '["openmetrics"]'
        ad.stackstate.com/ gitjob .init_configs: '[{}]'
        ad.stackstate.com/ gitjob .instances: |
          [ 
            {
              "prometheus_url": "http://%%host%%:8081/metrics",
              "namespace": "cattle_fleet_system", 
              "metrics": ["*"]
            }
          ]
      creationTimestamp: null
...
# This already exists in the pod spec, the container name needs to match the container that is exposing the openmetrics endpoint
spec:
  containers:
   - name:  gitjob  
   

[kkaempf]

@amolkharche13 this issue is marked as done (and I'm going to close it now). Please open a new one (referecing the old issue) with your request.

[amolkharche13]

@kkaempf Created https://github.com/rancher/fleet/issues/3648 , Thanks for your support.