Pods resources(CPU: request and limit) are not picking up even after adding resource preset

[yogesh-jha-miq]

Description: We have seen the SCDF document which says we can set the property resource preset to provide cpu & memory to the pods which SCDF brings up, we have used as below

server:
  metadata:
    annotations: <iam/role>
  image:
    registry: docker.io
    repository: bitnami/spring-cloud-dataflow
    tag: 2.11.5-debian-12-r2
    digest: ""
    pullPolicy: IfNotPresent
    pullSecrets: []
    debug: false
  composedTaskRunner:
    image:
      registry: docker.io
      repository: bitnami/spring-cloud-dataflow-composed-task-runner
      tag: 2.11.5-debian-12-r2
      digest: ""
  configuration:
    streamingEnabled: false
    batchEnabled: true
    accountName: spring-cloud-dataflow-preprod
    trustK8sCerts: false
    extraEnvVars:
      name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_MAXIMUM_CONCURRENT_TASKS'
      value: 50
  extraEnvVars:
    - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_PODANNOTATIONS'
      value: <value>
    - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_JOBANNOTATIONS'
      value: <value>
    - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_MAXIMUM_CONCURRENT_TASKS'
      value: "100"
    - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_IMAGEPULLPOLICY'
      value: "Always"
  podSecurityContext:
    enabled: true
    fsGroupChangePolicy: Always
    fsGroup: 0
  containerPorts:
    http: 8080
    jdwp: 5005
  replicaCount: 1
  updateStrategy:
    type: RollingUpdate
  startupProbe:
    enabled: false
    initialDelaySeconds: 120
    timeoutSeconds: 1
    periodSeconds: 20
    failureThreshold: 6
    successThreshold: 1
  livenessProbe:
    enabled: true
    initialDelaySeconds: 120
    timeoutSeconds: 1
    periodSeconds: 20
    failureThreshold: 6
    successThreshold: 1
  readinessProbe:
    enabled: true
    initialDelaySeconds: 120
    timeoutSeconds: 1
    periodSeconds: 20
    failureThreshold: 6
    successThreshold: 1
  networkPolicy:
    enabled: false
    allowExternal: false
    allowExternalEgress: false
  service:
    type: ClusterIP
    ports:
      http: 80
  ingress:
    enabled: true
    path: /*
    pathType: ImplementationSpecific
    hostname: <hostname>
    annotations:
      alb.ingress.kubernetes.io/certificate-arn: “<“arn:aws>”
      alb.ingress.kubernetes.io/group.name: campaign-management
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/scheme: internal
      alb.ingress.kubernetes.io/ssl-policy: <security policy>
      alb.ingress.kubernetes.io/target-type: ip
    tls: false
    certManager: false
    ingressClassName: <ingress>
  initContainers: []
  sidecars: []
  pdb:
    create: false
    minAvailable: ""
    maxUnavailable: ""
skipper:
  enabled: false
rabbitmq:
  enabled: false
mariadb:
  enabled: false
metrics:
  enabled: false
  pdb:
    create: false
externalDatabase:
  driver: org.mariadb.jdbc.Driver
deployer:
  podSecurityContext:
    enabled: true
    runAsUser: 0
    fsGroup: 0
  resourcesPreset: "xlarge"

what else values we need to set in this values.yaml file, Could anyone help here?

Release versions: we are installing "bitnami/spring-cloud-dataflow:2.11.5-debian-12-r2"

Screenshots: Where applicable, add screenshots to help explain your problem.

[cppwfs]

Here is a section in the documentation that discusses resourcesPreset: https://dataflow.spring.io/docs/installation/kubernetes/helm/#resource-requests-and-limits

[yogesh-jha-miq]

Hey @cppwfs In config map we are able to see the resource mapping as we have added the resource preset in values.yaml file but pods are not able to pick it up, Can you provide me a way to pass it as extraEnvVars variable similar to the below mentioned properties. extraEnvVars: - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_PODANNOTATIONS' value: - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_JOBANNOTATIONS' value: - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_MAXIMUM_CONCURRENT_TASKS' value: "100" - name: 'SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_KUBERNETES_ACCOUNTS_DEFAULT_IMAGEPULLPOLICY' value: "Always"

[cppwfs]

Just to verify did you follow the format shown here: https://github.com/bitnami/charts/blob/main/bitnami/common/templates/_resources.tpl#L13C13-L13C36?

[yogesh-jha-miq]

Hey @cppwfs I have tried hit & trial with these many configuration.

Is there any other annotation which we are missing?

Thank you.

[cppwfs]

Hello @yogesh-jha-miq , I removed the previous response, after we did some more digging.

Bitnami groups their props by prefix "server", "skipper", and "deployer" where the latter really means "tasks apps or stream apps".

Bitnami only supports resourcesPreset for tasks (dataflow i.e. 'server' configmap) but not for streams (skipper configmap). We will file issue w/ Bitnami to make this consistent and will provide the link to the issue once it is available.

NOTE: The resourcesPreset is completely a Bitnami only variable name. The final destination of these variables is always the Kubernetes "limits" or "requests".

We do discourage the use of "resourcesPreset" in favor using the Bitnami "resources" variable (i.e. 'deployer.resources').

[cppwfs]

Here is the issue reported to bitnami: https://github.com/bitnami/charts/issues/31301

[yogesh-jha-miq]

Thanks @cppwfs