GoogleContainerTools
failed to get filesystem from image when parent image is pulled from private insecure registry
Actual behavior While using a parent image hosted in an HTTP registry, any Dockerfile with any step would result in the following error:
error building image: error building stage: failed to get filesystem from image: gzip: invalid header
- A Dockerfile like this doesn't work
FROM private-reg:5000/busybox
RUN ls /
- A Dockerfile like this works
FROM busybox
RUN ls /
- A Dockerfile like this also works (but is useless)
FROM private-reg:5000/busybox
CMD ["/bin/sh"]
Expected behavior
That all of the above Dockerfiles work the same, regardless of registry.
To Reproduce Steps to reproduce the behavior:
- Push any image to private local http registry
- Run kaniko with this Dockerfile
FROM private-registry:5000/<any_image>
RUN ls /
Additional Information
- Dockerfile
FROM private-registry:5000/busybox
RUN ls /
- Build Context None
- Kaniko Image (fully qualified with digest)
gcr.io/kaniko-project/executor:latest sha256:6ecc43ae139ad8cfa11604b592aaedddcabff8cef469eda303f1fb5afe5e3034
gcr.io/kaniko-project/executor:debug sha256:fcccd2ab9f3892e33fc7f2e950c8e4fc665e7a4c66f6a9d70b300d7a2103592f
gcr.io/kaniko-project/executor:debug-v0.21.0 sha256:bce20d5d4a2d84d1cfd231ea0a3eca073c010b4f98fcdd844e4e4b88384dee59
gcr.io/kaniko-project/executor:v0.21.0 sha256:fee59f1fc71e70b3a0f4d93be747ff94a81e8079dcccef735005a29890b18a5e
Triage Notes for the Maintainers
| Description | Yes/No |
|---|---|
| Please check if this a new feature you are proposing |
|
| Please check if the build works in docker but not in kaniko |
|
Please check if this error is seen when you use --cache flag |
|
| Please check if your dockerfile is a multistage dockerfile |
|
EDIT: This issue does not happen when using an https registry (so without the insecure-pull but with skip-tls-verify-pull option).
The problem happens when the layers of the images pushed to the registry are not compressed. docker compresses these images automatically, so if you load your images from tars with docker load then kaniko works perfectly. But in order to load my images into my kubernetes cluster I'm forced to use containerd. And I do so using this command:
ctr -n=k8s.io images import private-reg:5000/busybox
This will not compress images, instead using the following works:
ctr -n=k8s.io images import --compress-blobs private-reg:5000/busybox
This workaround solves my problem, and I can use kaniko without a problem. But kaniko is still unable to handle uncompressed layers. Feel free to close this issue if you consider it solved.
