Image pull ignores insecure registries on containerd

[emilianosantucci]

Description

Using containerd as container runtime, docker pull image (and docker run) ignores the insecure registry defined on daemon.json. This makes unusable registry with HTTP or HTTPS with self-signed or invalid certificate.

Reproduce

1. Configure Docker to use containerd as container runtime (check Settings -> General -> Use containerd for pulling and storing images)
2. Define at least an insecure registry on ~/.docker/daemon.json or by Docker Desktop on Setting -> Docker Engine like below:

{
  "insecure-registries": [
    "my-insecure-registry.example.com"
  ]
}

3. Type into a terminal:

  docker pull my-insecure-registry.example.com/myimage:latest

Expected behavior

docker pull successfully store the image into local registry.

docker version

Client:
 Version:           27.2.0
 API version:       1.47
 Go version:        go1.21.13
 Git commit:        3ab4256
 Built:             Tue Aug 27 14:14:45 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.34.2 (167172)
 Engine:
  Version:          27.2.0
  API version:      1.47 (minimum version 1.24)
  Go version:       go1.21.13
  Git commit:       3ab5c7d
  Built:            Tue Aug 27 14:15:41 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.7.20
  GitCommit:        8fc6bcff51318944179630522a095cc9dbf9f353
 runc:
  Version:          1.1.13
  GitCommit:        v1.1.13-0-g58aa920
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    27.2.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.16.2-desktop.1
    Path:     /Users/emiliano/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.29.2-desktop.2
    Path:     /Users/emiliano/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.34
    Path:     /Users/emiliano/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Alpha) (Docker Inc.)
    Version:  v0.0.15
    Path:     /Users/emiliano/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/emiliano/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.25
    Path:     /Users/emiliano/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/emiliano/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.3.0
    Path:     /Users/emiliano/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/emiliano/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.13.0
    Path:     /Users/emiliano/.docker/cli-plugins/docker-scout

Server:
 Containers: 12
  Running: 1
  Paused: 0
  Stopped: 11
 Images: 15
 Server Version: 27.2.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.4-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 39.11GiB
 Name: docker-desktop
 ID: 554faa27-a357-4aa9-902e-dba63e850ac7
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/emiliano/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  my-insecure-registry.example.com
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

77121D55-FED5-4309-99F9-434930CBAEF6/20241006101557

Additional Info

Other issue talks about issue with Buildkit. I have tried to disable it but issue remains. The only way to currently execute docker pull with insecure registry is to do disable containerd as container runtime.

[sfrycertinia]

I have a similar issue configuring client certificates for a private registry. It works when not using containerd, by copying the certs.d from the users home to the VM it seems as per https://docs.docker.com/desktop/faqs/macfaqs/#add-client-certificates, however I can't find a way to make this work with containerd enabled.

[cFireworks]

At version v4.37.1，still have the same issue，Is there any way to deal with this problem?

[thompson-shaun]

👋 @cFireworks - Sorry to hear you're also impacted by this. Any chance you could provide some version/setup info (docker info && docker version) or perhaps a diagonostic ID?

Moby Menu > Troubleshoot > Get Support > Upload for Diagnostic ID docs ref

Thanks!

[lukaso]

This has affected me as well. The workaround is what I'm using since I don't need containerd particularly.

What I'm seeing is that the "WWW-Authenticate": Basic realm="${r.url}"`` response is not then leading to an "Authorization" header. So of course nothing works.

Here's my Diagnostic ID: 8CC53235-B216-4C59-AAA5-CC4EBEB74DBF/20250208163118

[riggzh]

same

[vvoland]

Please retest it in the next Docker Desktop release (v4.39) and let me know if the issue still persists.

[riggzh]

Please retest it in the next Docker Desktop release (v4.39) and let me know if the issue still persists.

The problem persists.

"insecure-registries": ["10.172.49.246"]

failed to do request: Head "https://10.172.49.246/v2/***/***/blobs/sha256:683586c0f2c0544b0395a9ac0fe0be9e811166d3c7cda05bb1994f3676c525f4": EOF

[vvoland]

The EOF seems rather unexpected, could you please provide the diagnostics after such pull failure?

[riggzh]

The EOF seems rather unexpected, could you please provide the diagnostics after such pull failure?

should use http not https with insecure-registries. It's work in contained set off

[jidaojiuyou]

same, how can i config containerd to enable insecure-registries?

[m00dawg]

Having this issue also, though I'm on Linux and am trying to push to ARM64.

docker buildx build --platform=linux/amd64,linux/arm64 . --push -t docker.redacted.com:5000/redacted
...
 > exporting to image:
------
failed to solve: rpc error: code = Unknown desc = failed to push docker.redacted.com:5000/redacted: failed to do request: Head "https://docker.redacted.com:5000/v2/redacted/blobs/sha256:b630ad74db3a139ece9bdad2f8644c9195cb5511ce5b12679b97ff1f570da0a7": http: server gave HTTP response to HTTPS client

I tried doing

docker buildx create --config=docker/buildkitd.toml 

with the contents of the buildkitd.toml being:

[registry."docker.redacted.com:5000"]
  http = true

And then my /etc/docker/daemon.json has:

{
  "insecure-registries":
    ["docker.redcated.com:5000"]
}

If I don't use buildx (and just use build) it works, but then I can't target linux/arm64.

[emilianosantucci]

@vvoland on v4.43.2 still persists.

[alebianchi96]

I've just met this problem on my Win11 system with docker v. 28.4.0

[LordFoom]

Ran into this one today: Fedora, docker v. 28.4.0

[jidaojiuyou]

Is it difficult to fix this? Or don't mind Mac users?