runc fails with permission denied (`mkdir /run/runc: permission denied`)

[khavish-bhundoo-cko]

Hello I am getting runc fails with permission denied when buildkit is parsing my Dockerfile for RUN commands while running buildkit in rootless mode

BUILDKITD_SOCKET=unix:///home/runner/run/buildkit/buildkitd.sock
rootlesskit buildkitd --root /home/runner/buildkitd --addr "$BUILDKITD_SOCKET"

Full error message: mkdir /run/runc: permission denied

Sample Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:7.0-alpine AS build
RUN dotnet --version

Buildctl

buildctl --addr unix:///home/runner/run/buildkit/buildkitd.sock build --frontend dockerfile.v0 --local context=. --local dockerfile=. --opt platform=linux/arm64,linux/amd64 --output type=image,"name=***.dkr.ecr.eu-west-1.amazonaws.com/org/repo:1.6.0-alpha.1",push=true --opt build-arg:VERSION=1.6.0-alpha.1

docker setup

Client:
 Version:           20.10.12
 API version:       1.41
 Go version:        go1.[16](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:17).12
 Git commit:        e91ed57
 Built:             Mon Dec 13 11:40:57 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.12
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.16.12
  Git commit:       459d0df
  Built:            Mon Dec 13 11:46:12 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.12
  GitCommit:        7b11cfaabd73bb80907dd23[18](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:19)2b9347b4245eb5d
 runc:
  Version:          1.0.2
  GitCommit:        v1.0.2-0-g52b36a2d
 docker-init:
  Version:          0.[19](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:20).0
  GitCommit:        de40ad0
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.24.7)

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: [20](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:21).10.12
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd[23](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:24)182b9347b4[24](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:25)5eb5d
 runc version: v1.0.2-0-g52b36a2d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.0-1049-aws
 Operating System: Ubuntu 20.04.4 LTS (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 36
 Total Memory: 92.19GiB
 Name: org-default-runners-github-runners-cxzdl-rxftg
 ID: LOIO:ZURY:DKJN:LRIU:2M[27](https://github.com/orgrepo/actions/runs/8295428168/job/22702441286#step:9:28):5DLY:YBPL:L7ZU:F7DV:ANAX:RTTF:FHBT
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: appdockerteam1
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

[AkihiroSuda]

What's your BuildKit version? Is this a recent regression?

dotnet

Is dotnet really relevant?

[khavish-bhundoo-cko]

@AkihiroSuda I am running buildkit 0.10.6. Running dotnet here is just a simple case that is easy to reproduce as well as being a common use case for multi-arch builds.

[ppmathis]

@khavish-bhundoo-cko I just stumbled across the same issue on a system where I manually deployed rootlesskit, buildkit, runc and slirp4netns from GitHub release builds. The fix in my case was to ensure that the directory /run/runc exists on the host system and that the user running rootlesskit buildkitd [...] has permissions to write in said directory.

An alternative would probably be to specify --root /tmp/runc to runc, as explained in https://github.com/opencontainers/runc?tab=readme-ov-file#rootless-containers, but I do not think this is possible as buildkit only offers a choice to specify a binary, but not any extra arguments.