testcontainers
[Bug]: Docs state docker context aware, but does not seem to work
Testcontainers version
v0.17.0
Using the latest Testcontainers version?
Yes
Host OS
OSX
Host arch
Arm64
Go version
1.19
Docker version
> docker version
Client: Docker Engine - Community
Version: 23.0.0
API version: 1.41 (downgraded from 1.42)
Go version: go1.19.5
Git commit: e92dd87c32
Built: Tue Jan 31 16:45:07 2023
OS/Arch: darwin/arm64
Context: colima
Server:
Engine:
Version: 20.10.20
API version: 1.41 (minimum version 1.12)
Go version: go1.18.7
Git commit: 03df974ae9e6c219862907efdd76ec2e77ec930b
Built: Wed Oct 19 02:58:31 2022
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: v1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: 5fd4c4d144137e991c4acebb2146ab1483a97925
docker-init:
Version: 0.19.0
GitCommit:
> colima version
colima version 0.5.2
git commit: 6b5b6fe0540e708f0c9d6e8919fab292c671fc72
runtime: docker
arch: aarch64
client: v23.0.0
server: v20.10.20
Docker info
> docker info
Client:
Context: colima
Debug Mode: false
Server:
Containers: 115
Running: 0
Paused: 0
Stopped: 115
Images: 5
Server Version: 20.10.20
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc io.containerd.runc.v2 io.containerd.runtime.v1.linux
Default Runtime: runc
Init Binary: docker-init
containerd version: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc version: 5fd4c4d144137e991c4acebb2146ab1483a97925
init version:
Security Options:
seccomp
Profile: default
Kernel Version: 5.15.82-0-virt
Operating System: Alpine Linux v3.16
OSType: linux
Architecture: aarch64
CPUs: 2
Total Memory: 1.93GiB
Name: colima
ID: VGCS:DVBU:ZQ3G:A75O:BZ5P:UNB7:7NJH:537I:2B7V:A2J5:KW4J:VH5A
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
What happened?
Starting a container via testcontainers returns the following error:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?: failed to create container
Relevant log output
No response
Additional information
states the following: https://github.com/testcontainers/testcontainers-go/blob/ba569c031b4084de461f74e168a02bd9f1553874/docs/system_requirements/using_colima.md?plain=1#L6-L8
The context is set correctly...
# docker context show
colima
I can work work around that, but based on the docs assumed this was supposed to work.
If testcontainers is not context aware, I guess this is a feature request & not a bug report 😄
I had to run sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock as a workaround, the simpler export DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock" did not work for some reason either (that might be a separate bug!)
Hi @rcrowe did you have the chance to test this with latest v0.23.0? We did some efforts to improve the Docker host & socket path discovery in the recent releases. Could you post here your findings?
@mdelapenya looks like it's still a problem with v0.23.0:
$ go test ./
2023/09/08 13:05:53 failed getting information about docker server: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
On https://github.com/testcontainers/testcontainers-go/commit/69b9fc4405a7e8763081def85102e2a695a06c1d it's even worse:
fatal error: sync: unlock of unlocked mutex
goroutine 35 [running]:
sync.fatal({0x104cd61e0?, 0x0?})
/Users/mossity/sdk/go1.21.0/src/runtime/panic.go:1061 +0x20
sync.(*Mutex).unlockSlow(0x105276e64, 0xffffffff)
/Users/mossity/sdk/go1.21.0/src/sync/mutex.go:229 +0x38
sync.(*Mutex).Unlock(0x0?)
/Users/mossity/sdk/go1.21.0/src/sync/mutex.go:223 +0x58
sync.(*Once).doSlow(0x8?, 0x105242260?)
/Users/mossity/sdk/go1.21.0/src/sync/once.go:76 +0x13c
sync.(*Once).Do(...)
/Users/mossity/sdk/go1.21.0/src/sync/once.go:65
github.com/testcontainers/testcontainers-go.(*DockerClient).Info(_, {_, _})
/Users/mossity/workspace/testcontainers-go/testcontainers-go/docker_client.go:39 +0x88
github.com/testcontainers/testcontainers-go.NewDockerClientWithOpts({0x104ec2350, 0x105276a20}, {0x0?, 0x0, 0x0})
/Users/mossity/workspace/testcontainers-go/testcontainers-go/docker_client.go:102 +0xb0
github.com/testcontainers/testcontainers-go.NewDockerProvider({0x14000368ec0, 0x2, 0x14000446150?})
/Users/mossity/workspace/testcontainers-go/testcontainers-go/provider.go:142 +0x10c
github.com/testcontainers/testcontainers-go.ProviderType.GetProvider(0x0, {0x140001f3210, 0x1, 0x1049bac68?})
/Users/mossity/workspace/testcontainers-go/testcontainers-go/provider.go:113 +0x48c
github.com/testcontainers/testcontainers-go.GenericContainer({_, _}, {{{{0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}, ...}, ...}, ...})
/Users/mossity/workspace/testcontainers-go/testcontainers-go/generic.go:126 +0xf8
testcontainers-example.TestWithRedis(0x14000184b60)
/Users/mossity/workspace/testcontainers-go/foo_test.go:18 +0x14c
testing.tRunner(0x14000184b60, 0x104eb9330)
/Users/mossity/sdk/go1.21.0/src/testing/testing.go:1595 +0xe8
created by testing.(*T).Run in goroutine 1
/Users/mossity/sdk/go1.21.0/src/testing/testing.go:1648 +0x33c
I think https://github.com/testcontainers/testcontainers-go/issues/1490 introduced a bug, I don't think swapping out the sync.Once like is being done in https://github.com/testcontainers/testcontainers-go/blob/69b9fc4405a7e8763081def85102e2a695a06c1d/docker_client.go#L43 is ever valid: https://go.dev/play/p/swQwZQBIWOD
It looks like the docker context logic lives exclusively in https://github.com/docker/cli, not https://github.com/docker/docker, so unfortunately not very easy to just support contexts directly. It seems like fixing the docs to not imply that contexts will work at all.
@mdelapenya I can confirm it's still not working as advertised without the symlink workaround when testing against v0.24.1
@benmoss https://github.com/testcontainers/testcontainers-go/issues/815#issuecomment-1711990246 this was already solved in v0.24.1, thanks for the report!
@rcrowe I need to properly setup my machine to use colima to test this, but I need to create room for working on other container runtimes, although not sure when 😞 . If you think updating the docs with the symlink workround is enough, please let's change it in the docs.
Can confirm this is still not working on v0.25. The only workaround is:
sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
So I think we can actively add that to the docs, right? Is anybody interested in the contribution?
After digging into the code, adding a client.FromEnv option here will make the DOCKER_HOST workaround viable. It should make the docker client aware of the four environmental variables stated here. If there aren't unwanted side effects that I'm not ware of, I can make a PR about this.
Oops, I was mistaken. DOCKER_HOST does work in v0.25.
But sshfs that colima VM uses doesn't seem to handle mounted sockets from host correctly (the other two, virtiofs and p9, don't even support stat syscall on sockets, so can't even mount sockets with them), and because testcontainers mounts the host docker socket to the reaper container, using custom docker host is not possible. Reaper container will have to use the default docker socket in the colima VM, which is not mounted from host machine, to function. Tested with colima v0.6.6.
So linking the docker socket to the default path is the only possible way for reaper to work. If disable reaper, DOCKER_HOST will work.
Hi folks, could any of you update the docs for Colima? I'd need to create a local environment, which is something I cannot do at the moment, so I'd appreciate your assistance here 🙏
@mdelapenya Try sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
I had been facing similar issues. Turned out, I was looking at the wrong documentation. On checking documentation for Java, I found colima had to be started with flag --network-address for this to work well. Here are the steps I followed:
sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
colima stop
colima start --network-address
export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock
export TESTCONTAINERS_HOST_OVERRIDE=$(colima ls -j | jq -r '.address')
export DOCKER_HOST="unix://${HOME}/.colima/default/docker.sock"
I've tried running the example code, and the errors have been fixed. Hope this helps!
Edit: Missed adding the relevant link
