File changes done with RUN using paths including symlinks are ignored

[venushka]

Actual behavior Files created or attributes modified in RUN instructions in the Dockerfile is ignored if the path to the file includes a symlink.

Assume /var/run is a symlink pointing at /run, and we create a new file touch /var/run/test. This file does not end up in the image, however touch /run/test works. When using base images, if upstream image paths changes to a symlink, images that refer to paths via symlinks will not work as expected.

Expected behavior Any changes to the filesystem via a RUN instruction must be included in the image.

To Reproduce Steps to reproduce the behavior:

1. Create this Dockerfile

FROM openresty/openresty:alpine
RUN mkdir -p /var/run/openresty \
  && chown -R nobody:nobody /var/run/openresty \
  && touch /var/run/test
USER nobody

2. Build it with Docker

docker build -t built-with-docker .

3. Check for the new file /var/run/test created and ownership of /var/run/openresty changed to nobody:nobody

$ docker run -it built-with-docker:latest ls -l /var/run/
total 4
drwxr-xr-x    1 nobody   nobody        4096 Dec 12 20:08 openresty
-rw-r--r--    1 root     root             0 Jan  5 17:44 test

4. Build it with Kaniko and load the image to Docker cache

docker run \
  -v `pwd`:/workspace \
  gcr.io/kaniko-project/executor:debug \
  --dockerfile "./Dockerfile" --destination "built-with-kaniko:latest" --context dir:///workspace/ \
  --insecure --insecure-pull --no-push  \
  --tarPath=/workspace/image.tar.gz --snapshotMode=full

docker load --input ./image.tar.gz

5. Check for the new file created and file ownership change.

$ docker run -it built-with-kaniko:latest ls -l /var/run/
total 4
drwxr-xr-x    2 root     root          4096 Dec 12 20:08 openresty

Additional Information

• Dockerfile Included in the reproduction steps above.
• Build Context No other files needed
• Kaniko Image (fully qualified with digest) gcr.io/kaniko-project/executor:debug (ffca8c9f01a2)

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing	[ ]
Please check if the build works in docker but not in kaniko	[x]
Please check if this error is seen when you use --cache flag	[x]
Please check if your dockerfile is a multistage dockerfile	[ ]

[matthew-muscat]

I've also run into this issue — in my case, i was creating an empty directory

FROM registry.access.redhat.com/ubi8/ubi:latest

RUN mkdir -p /var/run/nginx
    && mkdir -p /var/cache/nginx

Running the above build via Kaniko will result in a container with the following properties...

1. The /var/run/nginx directory will NOT be present in the container (due to the image using a symlink where /var/run -> /run)
2. The /var/cache/nginx directory will be present in the container, as this is standard FS path

Based on this, it would appear that new files / folders created via a RUN command that involve a symlink in the path are resulting in missing changes on FS snapshots

[MrMage]

This might be related to #1547.

[jjanuszkiewicz]

I'm also hitting this. In my case /var/run is a symlink to run and this command doesn't end up affecting the final image:

RUN touch /var/run/nginx.pid && \
    chown nginx /var/run/nginx.pid

but this one does:

RUN touch /run/nginx.pid && \
    chown nginx /run/nginx.pid