docker
--build-args/ARG does not overwrite ENV variables of the same name from a base image
Description
ARG variables of the same name as an environment variable set in a base image do not overwrite the value of the variable during build time. Is it intentional for the primary environment to take precedence over the temporary environment? This is especially irritating when a base image sets a proxy value to empty and the child image needs to temporarily set a build environment that uses a different proxy value than the parent image.
Steps to reproduce the issue:
- Build a parent image and set a variable with ENV
- Base a child image off the built parent and apply an ARG of the same name as the parent ENV var.
- Echo the ARG in a dockerfile RUN directive in the child
- Build the child. Note that the ENV variable of the parent is echo'd not the temporary ARG value
Describe the results you received:
Create parent dockerfile and set an ENV variable.
# parent.dockerfile
FROM alpine:latest
ENV HELLO="World"
Build the parent image
❯ docker build -t parent:latest -f parent.dockerfile .
Create a child dockerfile and set a build-arg with the same name as the ENV var from parent.
# child.dockerfile
FROM parent:latest
ARG HELLO
RUN echo "HELLO=${HELLO}"
Build the child image.
❯ docker build -t child:latest --build-arg HELLO=Earth -f child.dockerfile --no-cache .
Sending build context to Docker daemon 3.072kB
Step 1/4 : FROM parent:latest
---> b319a2900116
Step 2/4 : ARG HELLO
---> Running in df26b87ef593
Removing intermediate container df26b87ef593
---> 037665231ae3
Step 3/4 : RUN echo "HELLO=${HELLO}"
---> Running in b79b3944af01
HELLO=World
Removing intermediate container b79b3944af01
---> 6772fb18124d
Step 4/4 : CMD [ "echo", "HELLO=${HELLO}" ]
---> Running in 8f7fc5bc845b
Removing intermediate container 8f7fc5bc845b
---> fdaed33c1026
Successfully built fdaed33c1026
Successfully tagged child:latest
Result: In Step 3/4 the value echo'd is that of the original ENV var in the parent.
Describe the results you expected:
I would expect that the temporary environment would take precedence over the core environment during build and echo would produce HELLO=Earth in the example above.
Additional information you deem important (e.g. issue happens only occasionally):
Output of docker version:
Client:
Cloud integration: 1.0.17
Version: 20.10.8
API version: 1.41
Go version: go1.16.6
Git commit: 3967b7d
Built: Fri Jul 30 19:55:20 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.8
API version: 1.41 (minimum version 1.12)
Go version: go1.16.6
Git commit: 75249d8
Built: Fri Jul 30 19:52:10 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.9
GitCommit: e25210fe30a0a703442421b0f60afac609f950a3
runc:
Version: 1.0.1
GitCommit: v1.0.1-0-g4144b63
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
buildx: Build with BuildKit (Docker Inc., v0.6.1-docker)
compose: Docker Compose (Docker Inc., v2.0.0-rc.3)
scan: Docker Scan (Docker Inc., v0.8.0)
Server:
Containers: 6
Running: 3
Paused: 0
Stopped: 3
Images: 67
Server Version: 20.10.8
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: e25210fe30a0a703442421b0f60afac609f950a3
runc version: v1.0.1-0-g4144b63
init version: de40ad0
Security Options:
seccomp
Profile: default
Kernel Version: 5.10.47-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.774GiB
Name: docker-desktop
ID: BSL6:V5BU:UHMQ:5ZQ7:QELY:VD6D:A7U5:434O:7WSO:FHHJ:SAMI:UONQ
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http://proxy-lmi.global.lmco.com:80
HTTPS Proxy: http://proxy-lmi.global.lmco.com:80
No Proxy: 127.0.0.10/8,localhost,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.lmco.com
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Additional environment details (AWS, VirtualBox, physical, etc.):
This is a known limitation of the classic (non-buildkit) builder; with work in progress to make BuildKit the default builder (for Docker Desktop, this is already the case, but not yet for Linux installs), I'm not sure if this will be addressed.
Here's the example that's provided above, but with BuildKit used;
DOCKER_BUILDKIT=1 docker build -t parent:latest -f parent.dockerfile .
[+] Building 0.1s (5/5) FINISHED
=> [internal] load build definition from parent.dockerfile 0.0s
=> => transferring dockerfile: 83B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> CACHED [1/1] FROM docker.io/library/alpine:latest 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:04ee7e261e91f63c023b9d72dc33890c2ec33845ef07e3d8ba9b5bb8484ec256 0.0s
=> => naming to docker.io/library/parent:latest 0.0s
Then, building the "child" image (I'm using --progress=plain to show the output of each step;
DOCKER_BUILDKIT=1 docker build -t child:latest --progress=plain --build-arg HELLO=Earth -f child.dockerfile --no-cache .
#1 [internal] load build definition from child.dockerfile
#1 transferring dockerfile:
#1 transferring dockerfile: 101B done
#1 DONE 0.0s
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/library/parent:latest
#3 DONE 0.0s
#4 [1/2] FROM docker.io/library/parent:latest
#4 CACHED
#5 [2/2] RUN echo "HELLO=Earth"
#5 0.304 HELLO=Earth
#5 DONE 0.3s
#6 exporting to image
#6 exporting layers 0.1s done
#6 writing image sha256:6d2aa4b9d3e6a05b1341f66a5af77631e7b21bc43aad18c50491d565d4160074
#6 writing image sha256:6d2aa4b9d3e6a05b1341f66a5af77631e7b21bc43aad18c50491d565d4160074 done
#6 naming to docker.io/library/child:latest done
#6 DONE 0.1s
If possible, I'd recommend using BuildKit for your builds (easiest way to enable that is to set DOCKER_BUILDKIT=1 in your shell, or add it to your shell's profile)
Ah understood and thank you. Unfortunately, due to the environment we use specifically we can't use build kit because of the x503 self signed certificate issue with build kit and private registries. Good to know this particular issue is solved though going forward.
Unfortunately, due to the environment we use specifically we can't use build kit because of the x503 self signed certificate issue with build kit and private registries.
Hmm.. good one; I recall this being more of an issue when using the docker-container driver, but perhaps I'm mistaken; @crazy-max can you fill me in on that one? (are there still issues to fix for that?)
I met the issue too on Docker version 19.03.13, build 4484c46
Used DOCKER_BUILDKIT=1 to solve the problem.
closing this one, as BuildKit is now used as default for Linux containers, and it's unlikely to be fixed for the legacy builder.