Unable to relocate Docker data directory to an external volume/drive - rename/cross-device link error thrown

[connorblack]

Description

Attempted to relocate the Docker data path to an external drive using the Docker Desktop application by selecting a new location. Docker throws a runtime error and then reverts back to the default location after restarting:

running callback: failed to rename VM disk: rename /Users/<username>/Library/Containers/com.docker.docker/Data/vms/0/data /Volumes/eSSD/DockerDesktop: cross-device link

As a workaround, I tried the following steps:

1. Rsyncing files in com.docker.docker:

   rsync -avz --progress --exclude='*.sock' --exclude='*log.*' --exclude='*.raw' /Users/<username>/Library/Containers/com.docker.docker/ /Volumes/eSSD/com.docker.docker/
   

2. Manually updating the dataFolder in settings.json: Updated the "dataFolder" field in the settings.json file located at /Users/<username>/Library/Group Containers/group.com.docker/settings.json.

While this workaround seems to work somewhat consistently, I still encounter issues where Visual Studio Code and other applications are unable to connect to Docker after the relocation. Socket daemons also hang.

Reproduce

1. (Optional) Synchronize the Docker data directory to an external SSD using the provided rsync command.
2. Update the "dataFolder" field in settings.json to the new path on the SSD.
3. Attempt to use Docker with Visual Studio Code or other applications.

Expected behavior

1. Select new path via GUI
2. Good to go!

docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.30.0 (149282)
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:48:04 2024
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    26.1.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0-desktop.1
    Path:     /Users/connorblack/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.27.0-desktop.2
    Path:     /Users/connorblack/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.29
    Path:     /Users/connorblack/.docker/cli-plugins/docker-debug
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/connorblack/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.23
    Path:     /Users/connorblack/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.4
    Path:     /Users/connorblack/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.1.0
    Path:     /Users/connorblack/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/connorblack/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.8.0
    Path:     /Users/connorblack/.docker/cli-plugins/docker-scout

Server:
 Containers: 2
  Running: 0
  Paused: 0
  Stopped: 2
 Images: 2
 Server Version: 26.1.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.6.26-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 16
 Total Memory: 125.1GiB
 Name: docker-desktop
 ID: 52908e3f-7bca-42d8-ad4c-217654857cfd
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/connorblack/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

8BC59B8A-21DC-418F-A50E-AFE7E330CE13/20240602232804

Additional Info

MacBook Pro 16-inch, Nov 2023 Chip Apple M3 Max Memory 128 GB Startup disk Macintosh HD macOS 14.5 (23F79)

[nesanakin]

Just voicing that I am having the same issue on MacOS 12.7.1 ("not officially supported" etc, but still). The workaround given here doesn't work for me–Docker Desktop just hangs on startup and can only be killed by kill -9ing all the docker processes. I also tried just symlinking ~/Library/Containers/com.docker.docker to the external drive, instead of changing the path in settings.json, with the same results.

I did get it to work by first moving the Docker disk image to another location on the same drive using the GUI. Then, I can move the DockerDesktop/Docker.raw file to an external drive and replace it with a single symlink. The size of the image is limited to the size of the main drive because the GUI doesn't detect that it's physically located somewhere else with more space. But it does work!