1.20.0 / latest not working correctly - file should be deleted, but is still in the final image (Build via Gitlab)

[linuxmail]

Actual behavior

• Dockerfile

...
RUN  rm /usr/local/etc/php-fpm.d/zz-docker.conf \
     && find $APP_HOME -type f -print0 | xargs -0 chmod 644 \
     && find $APP_HOME -type d -print0 | xargs -0 chmod 755 \
...

ls /usr/local/etc/php-fpm.d/zz-docker.conf 
/usr/local/etc/php-fpm.d/zz-docker.conf
~ $ cat  /usr/local/etc/php-fpm.d/zz-docker.conf 
[global]
daemonize = no

[www]
listen = 9000

Expected behavior File should be deleted in final image, but

~ $ cat  /usr/local/etc/php-fpm.d/zz-docker.conf 
cat: can't open '/usr/local/etc/php-fpm.d/zz-docker.conf': No such file or directory

To Reproduce

• Use

image:
  name: gcr.io/kaniko-project/executor:debug
  entrypoint: [""]
...

# Set defaults for Kaniko build image
variables:
  KANIKO_CACHE_ARGS: "--cache=true --compressed-caching=false"
  KANIKO_SNAPSHOT_ARGS: "--use-new-run --cleanup"
  VERSIONLABELMETHOD: "OnlyIfThisCommitHasVersion" # options: "OnlyIfThisCommitHasVersion","LastVersionTagInGit"
  IMAGENAME: $DOCKER_IMAGE_NAME
  DOCKERFILE: Dockerfile
  BRANCH: $RELEASE_BRANCH
  DOCKER_BUILD_CONTEXT: ${CI_PROJECT_DIR}/

Additional Information

...
INFO[0040] Taking snapshot of files...                  
INFO[0041] RUN  rm /usr/local/etc/php-fpm.d/zz-docker.conf      && find $APP_HOME -type f -print0 | xargs -0 chmod 644      && find $APP_HOME -type d -print0 | xargs -0 chmod 755      && chmod 644 /usr/local/etc/php-fpm.d/www.conf      && chmod 755 $APP_HOME/bin/* /var/lib/nginx/      && chown -R app:app $APP_HOME /var/lib/nginx/tmp      && chmod +x entrypoint.sh      && mkdir -p /var/lib/nginx/logs      && ln -sf /dev/stderr /var/lib/nginx/logs/error.log      && update-ca-certificates 
INFO[0041] Cmd: /bin/sh                                 
INFO[0041] Args: [-c rm /usr/local/etc/php-fpm.d/zz-docker.conf      && find $APP_HOME -type f -print0 | xargs -0 chmod 644      && find $APP_HOME -type d -print0 | xargs -0 chmod 755      && chmod 644 /usr/local/etc/php-fpm.d/www.conf      && chmod 755 $APP_HOME/bin/* /var/lib/nginx/      && chown -R app:app $APP_HOME /var/lib/nginx/tmp      && chmod +x entrypoint.sh      && mkdir -p /var/lib/nginx/logs      && ln -sf /dev/stderr /var/lib/nginx/logs/error.log      && update-ca-certificates] 
INFO[0041] Running: [/bin/sh -c rm /usr/local/etc/php-fpm.d/zz-docker.conf      && find $APP_HOME -type f -print0 | xargs -0 chmod 644      && find $APP_HOME -type d -print0 | xargs -0 chmod 755      && chmod 644 /usr/local/etc/php-fpm.d/www.conf      && chmod 755 $APP_HOME/bin/* /var/lib/nginx/      && chown -R app:app $APP_HOME /var/lib/nginx/tmp      && chmod +x entrypoint.sh      && mkdir -p /var/lib/nginx/logs      && ln -sf /dev/stderr /var/lib/nginx/logs/error.log      && update-ca-certificates] 
INFO[0042] Pushed git.example.com:5555/backoffice-tools/client-portal-api/cache@sha256:8f3f4c9cb457995f1135f40b1361c74a5435837cf043c2879f88aa49770d2a4d 
INFO[0043] Taking snapshot of files...       
...

Set the version to executor:v1.19.2-debug solved the issue.

[aaron-prindle]

Thank you for flagging this @linuxmail and providing the repro info. This is likely an issue with the changes made by the recently merged PR here - https://github.com/GoogleContainerTools/kaniko/pull/2943. Currently trying to repro and understand exactly how this is occurring as well as the cases this occurs for. We might need to rollback the PR mentioned to resolve this issue