actions-runner-controller actions-runner-dind-rootless image not built for runner version 2.317.0

Checks

[X] I've already read https://github.com/actions/actions-runner-controller/blob/master/TROUBLESHOOTING.md and I'm sure my issue is not covered in the troubleshooting guide.
[X] I'm not using a custom entrypoint in my runner image

Controller Version

v0.27.6

Helm Chart Version

No response

CertManager Version

No response

Deployment Method

Helm

cert-manager installation

N/A

Checks

[X] This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
[X] I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
[X] My actions-runner-controller version (v0.x.y) does support the feature
[X] I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
[X] I've migrated to the workflow job webhook event (if you using webhook driven scaling)

Resource Definitions

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  creationTimestamp: "2023-08-31T16:13:22Z"
  generation: 9276
  name: gh-terraform-secure
  namespace: actions-runner-system
  resourceVersion: "137974423"
  uid: 6a78ee69-9a07-4ac6-9d7d-3b93dfe195fe
spec:
  effectiveTime: "2024-06-10T13:03:25Z"
  replicas: 0
  selector: null
  template:
    metadata:
      annotations:
        ad.datadoghq.com/tags: '{"gha_runner_type": "gh-terraform-secure"}'
      labels:
        app.kubernetes.io/name: gh-terraform-secure
    spec:
      dockerdContainerResources: {}
      env:
      - name: DISABLE_RUNNER_UPDATE
        value: "false"
      - name: DISABLE_WAIT_FOR_DOCKER
        value: "true"
      ephemeral: true
      image: xxxxxxxxxxx.dkr.ecr.eu-west-1.amazonaws.com/tools/gh-terraform-runner:v0.1.73
      labels:
      - gh-terraform-secure
      organization: jumo
      resources:
        requests:
          cpu: "1"
          memory: 512Mi
      serviceAccountName: gh-terraform-secure-runner
      tolerations:
      - effect: NoSchedule
        key: runner-type
        operator: Equal
        value: gh-terraform-secure

To Reproduce

Go to: https://hub.docker.com/r/summerwind/actions-runner-dind-rootless/tags

Latest tag is v2.316.0-ubuntu-20.04-49490c4

Go to: https://hub.docker.com/r/summerwind/actions-runner-dind/tags

Latest tag is v2.317.0-ubuntu-20.04-8947327

Describe the bug

Latest image build (with runner version 2.317.0) for actions-runner-dind-rootless failed (for both ubuntu-20.04 and ubuntu-22.04): https://github.com/actions-runner-controller/releases/actions/runs/9415130587/job/25935482037

Describe the expected behavior

I would expect an image would be available with runner version 2.317.0, same as for the other images built from this repository.

Whole Controller Logs

N/A

Whole Runner Pod Logs

√ Settings Saved.

2024-06-10 13:29:13.952  DEBUG --- Runner successfully configured.
{
  "agentId": 1044721,
  "agentName": "gh-terraform-hsr79-5gf28",
  "poolId": 1,
  "poolName": "Default",
  "disableUpdate": true,
  "ephemeral": true,
  "serverUrl": "https://pipelinesghubeus21.actions.githubusercontent.com/xxxxxxx/",
  "gitHubUrl": "https://github.com/jumo",
  "workFolder": "/runner/_work"
2024-06-10 13:29:13.955  NOTICE --- Docker wait check skipped. Either Docker is disabled or the wait is disabled, continuing with entrypoint
2024-06-10 13:29:13.956  NOTICE --- WARNING LATEST TAG HAS BEEN DEPRECATED. SEE GITHUB ISSUE FOR DETAILS:
2024-06-10 13:29:13.957  NOTICE --- https://github.com/actions/actions-runner-controller/issues/2056
}
√ Connected to GitHub

Current runner version: '2.316.0'
2024-06-10 13:29:16Z: Listening for Jobs
An error occured: Runner version v2.316.0 is deprecated and cannot receive messages.
Runner listener exit with terminated error, stop the service, no retry needed.
Exiting runner...
2024-06-10 13:29:16.469  NOTICE --- Runner init exited. Exiting this process with code 0 so that the container and the pod is GC'ed Kubernetes soon.

Additional Context

Jun 10 '24 14:06 mantoine96

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

Jun 10 '24 14:06 github-actions[bot]

Check the image used on ephemeralrunnersets kubectl get ephemeralrunnersets.actions.github.com -o jsonpath='{.items[].spec.ephemeralRunnerSpec.spec.containers[].image}'

Must be ghcr.io/actions/actions-runner:2.317.0 or ghcr.io/actions/actions-runner:latest

imagePullPolicy If image tag is latest ensure that imagePullPolicy is Always

Seem that default value of imagePullPolicy is IfNotPresent that can be the root cause of this issue ...

Jun 12 '24 08:06 YvesEarnix

@YvesZelros This isn't relevant to this discussion. The issue at hands here is that the latest build for the image for summerwind/actions-runner-dind-rootless containing version 2.317.0 of the GHA runner failed, due to:

#26 2.180 + PATH=/home/runner/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/runner/.local/bin:/home/runner/bin /home/runner/bin/dockerd-rootless-setuptool.sh install --skip-iptables
#26 2.199 [rootlesskit:parent] error: failed to start the child: fork/exec /proc/self/exe: operation not permitted
#26 2.200 [ERROR] RootlessKit failed, see the error messages and https://rootlesscontaine.rs/getting-started/common/ .
#26 ERROR: process "/bin/sh -c export SKIP_IPTABLES=1     && curl -fsSL https://get.docker.com/rootless | sh     && /home/runner/bin/docker -v" did not complete successfully: exit code: 1

#12 [linux/arm64  2/18] RUN apt-get update -y     && apt-get install -y software-properties-common     && add-apt-repository -y ppa:git-core/ppa     && apt-get update -y     && apt-get install -y --no-install-recommends     build-essential     curl     ca-certificates     dnsutils     ftp     git     iproute2     iputils-ping     iptables     jq     libunwind8     locales     netcat     net-tools     openssh-client     parallel     python3-pip     rsync     shellcheck     software-properties-common     sudo     telnet     time     tzdata     uidmap     unzip     upx     wget     zip     zstd     && ln -sf /usr/bin/python3 /usr/bin/python     && ln -sf /usr/bin/pip3 /usr/bin/pip     && rm -rf /var/lib/apt/lists/*
#12 CANCELED

This issue's goal is to figure out what needs to happen to fix this. In this case, it looks like there might have been an issue with the seccomp of the container used by BuildKit. This could be a transient issue, although I'm inclined to say it isn't given the build before that also failed with the same errors: https://github.com/actions-runner-controller/releases/actions/runs/9076975144/job/25047042394

Jun 12 '24 08:06 mantoine96

This seems likend to https://github.com/docker/docker-install/issues/417.

Downgrading the Docker version fixes the issue.

Jun 12 '24 10:06 mohamed-haidara-cko

I just got this to build without downgrading the docker version. Sadly it needs the --force flag.

# Install Docker rootless manually (non-interactive)
RUN curl -fsSL https://get.docker.com/rootless > rootless && \
    chmod +x ./rootless
    
RUN export SKIP_IPTABLES=1; \
    ./rootless --force

RUN /home/runner/bin/docker -v

I do however now get a different error when trying to run docker...

$ docker run hello-world

produces:

command -v selinuxenabled

exec rootlesskit --state-dir=/run/user/1000/dockerd-rootless --net=slirp4netns --mtu=65520 --slirp4netns-sandbox=auto --slirp4netns-seccomp=auto --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run --propagation=rslave /home/runner/bin/dockerd-rootless.sh --config-file /home/runner/.config/docker/daemon.json WARN[0000] [rootlesskit:parent] The host root filesystem is mounted as "master:33". Setting child propagation to "rslave" is not supported. [rootlesskit:parent] error: failed to setup UID/GID map: newuidmap 33 [0 1000 1 1 100000 65536] failed: newuidmap: write to uid_map failed: Operation not permitted : exit status 1

Jan 19 '25 10:01 Lewiscowles1986