actions-runner-controller icon indicating copy to clipboard operation
actions-runner-controller copied to clipboard

Runner not picking up the jobs due to SSL error (PartialChain)

Open mathwro opened this issue 1 year ago • 1 comments

Checks

  • [X] I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
  • [X] I am using charts that are officially provided

Controller Version

0.9.3

Deployment Method

Helm

Checks

  • [X] This isn't a question or user support case (For Q&A and community support, go to Discussions).
  • [X] I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Deployed AKS cluster (done through terraform)
2. Rolled helm charts on (done through terraform)
3. Verify connection is established to github
4. Start github actions job on runner
5. Nothing is happening

Describe the bug

We are running a github enterprise server (version 3.12.4) which is hosted in Azure as well. It's configured as such that we need a custom self-signed certificate to properly authenticate.

The controller and listener pod is spinning up without showing errors.

The runner pod doesn't seem to be scaling, but it's spinning up according to the minimum amount of pods according to the helm configuration. However i can see that it sees that a job is available and tries to pick it up, but failing due to a PartialChain error when trying to establish SSL connection.

If i run a bash to the runner pod it self, and execute the "run.sh" script it does pick up the job and begin running it.

Describe the expected behavior

Runner will automatically pick up the available jobs and start running them.

Additional Context

gha-runner-scale-set:

  set {
    name  = "githubConfigSecret.github_token"
    value = var.github_token
  }
  set {
    name  = "githubConfigUrl"
    value = "https://github.<serverurl>.com/<company>
  }
  set {
    name  = "controllerServiceAccount.namespace"
    value = "arc-controller"
  }
  set {
    name  = "controllerServiceAccount.name"
    value = local.controller_serviceAccount
  }
  set {
    name  = "githubServerTLS.certificateFrom.configMapKeyRef.name"
    value = kubernetes_config_map.trustedca.metadata.0.name
  }
  set {
    name  = "githubServerTLS.certificateFrom.configMapKeyRef.key"
    value = "${local.certificate_key}.crt"
  }
  set {
    name  = "githubserverTLS.runnerMountPath"
    value = "/etc/ssl/certs/"
  }
  set {
    name  = "minRunners"
    value = var.min_pods
  }
  set {
    name  = "maxRunners"
    value = var.max_pods
  }

Controller Logs

https://gist.github.com/mathwro/e328c959be2213fc876e21443fa8cf55

Runner Pod Logs

https://gist.github.com/mathwro/1b027ac1192a3a5870955840c177023a

mathwro avatar Jul 09 '24 08:07 mathwro

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

github-actions[bot] avatar Jul 09 '24 08:07 github-actions[bot]

Hey @mathwro,

Did you manage to resolve the issue? From the description, it seems to be network connectivity issue, but I just want to double-check.

nikola-jokic avatar May 07 '25 13:05 nikola-jokic

I'll close this issue since it seems like it was related to network connectivity.

nikola-jokic avatar May 15 '25 10:05 nikola-jokic

Sorry, i missed the comment last week. We ended up getting it solved, and was an issue related to the certificates being used since we were using self-signed and some information was missing in the ones we were testing with.

mathwro avatar May 15 '25 10:05 mathwro

Thank you for letting us know! I'm glad you resolved it!

nikola-jokic avatar May 15 '25 10:05 nikola-jokic