Setup CI test that uses kind in GH actions and perform one (smaller) benchmark.

[bwplotka]

prombench supports kind but we never test it on CI, nor not many use it. To make that path healthy I propose we leverage it to test prombench in general -- to deploy cluster, infra resources and deploy one small benchmark (with reduced amount of metrics), all within 5m timeout on GH Actions. Ambitious? Maybe (:

Acceptance Criteria

• On every PR we will create a tiny kind, deploy all infra resources, and one smaller benchmark and expect all resources to be successful, then tear down.

[Vandit1604]

Hey, I would like to work on this!

[Vandit1604]

I got a prototype going on and have some questions:

name: Prombench on Kind CI

on:
  pull_request:
    branches:
      - '**'

jobs:
  prombench-kind:
    name: Setup Prombench on Kind
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
    - name: Checkout Code
      uses: actions/checkout@v3

    - name: Setup Go
      uses: actions/setup-go@v4
      with:
        go-version: 1.20

    - name: Build Infra CLI Tool
      run: |
        cd infra/
        go build .

    - name: Create Kind Cluster
      env:
        CLUSTER_NAME: prombench
        PR_NUMBER: ${{ github.event.pull_request.number }}
      run: |
        ../infra/infra kind cluster create -v PR_NUMBER:$PR_NUMBER \
            -v CLUSTER_NAME:$CLUSTER_NAME -f manifests/cluster_kind.yaml

    - name: Remove Control-Plane Taint
      env:
        CLUSTER_NAME: prombench
      run: |
        kubectl --context kind-$CLUSTER_NAME taint nodes $CLUSTER_NAME-control-plane node-role.kubernetes.io/control-plane-

    - name: Apply Cluster Infra Resources
      env:
        CLUSTER_NAME: prombench
        GRAFANA_ADMIN_PASSWORD: password
        DOMAIN_NAME: prombench.prometheus.io
        OAUTH_TOKEN: ""
        WH_SECRET: ""
        GITHUB_ORG: prometheus
        GITHUB_REPO: prometheus
        SERVICEACCOUNT_CLIENT_EMAIL: [email protected]
      run: |
        ../infra/infra kind resource apply -v CLUSTER_NAME:$CLUSTER_NAME \
            -v DOMAIN_NAME:$DOMAIN_NAME \
            -v GRAFANA_ADMIN_PASSWORD:$GRAFANA_ADMIN_PASSWORD \
            -v OAUTH_TOKEN="$(printf $OAUTH_TOKEN | base64 -w 0)" \
            -v WH_SECRET="$(printf $WH_SECRET | base64 -w 0)" \
            -v GITHUB_ORG:$GITHUB_ORG \
            -v GITHUB_REPO:$GITHUB_REPO \
            -v SERVICEACCOUNT_CLIENT_EMAIL:$SERVICEACCOUNT_CLIENT_EMAIL \
            -f manifests/cluster-infra

    - name: Deploy Small Benchmark
      env:
        CLUSTER_NAME: prombench
        PR_NUMBER: ${{ github.event.pull_request.number }}
        RELEASE: main
        DOMAIN_NAME: prombench.prometheus.io
        GITHUB_ORG: prometheus
        GITHUB_REPO: prometheus
      run: |
        ../infra/infra kind resource apply -v CLUSTER_NAME:$CLUSTER_NAME \
            -v PR_NUMBER:$PR_NUMBER -v RELEASE:$RELEASE -v DOMAIN_NAME:$DOMAIN_NAME \
            -v GITHUB_ORG:$GITHUB_ORG -v GITHUB_REPO:$GITHUB_REPO \
            -f manifests/prombench/benchmark

    - name: Wait for Benchmark to Stabilize
      run: |
        kubectl wait --for=condition=ready pod --all --timeout=300s

    - name: Tear Down Cluster
      if: always()
      env:
        CLUSTER_NAME: prombench
      run: |
        kind delete cluster

1. I'm building the infra tool in the workflow. Is that fine?
2. When you say “smaller” benchmark, does that mean, Less number of components used or fewer things done but all surface area covered? I am not confident in the benchmark part of the issue, but I can learn.

According to the answer of the above (2nd) question. I'll raise a pull request, After I write the benchmark YAML files,