docker
Occasionally failed commit on ref
Contributing guidelines
- [X] I've read the contributing guidelines and wholeheartedly agree
I've found a bug and checked that ...
- [X] ... the documentation does not mention anything about my problem
- [X] ... there are no open or closed issues that are related to my problem
Description
When using buildx to build a multi-architecture image, an error occurs: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6
Expected behaviour
Stable build succeeds
Actual behaviour
docker buildx build --platform linux/amd64 linux/arm64 --labels commit_id=xxxx --tags jdb11 .
first failure:
#16 [linux/amd64 jdk11 1/1] FROM docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:c16642dc269cd208adb76c0d9efe16098d0b47677f767e2c66eeb3bd9008a918
#16 sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2 33.55MB / 37.15MB 210.2s
#16 sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2 35.65MB / 37.15MB 212.0s
#16 DONE 248.9s
#32 [linux/amd64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
#32 ERROR: failed commit on ref "layer-sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2": unexpected commit digest sha256:9bc7b10ff3793ea67fd1628262aafe7e24319ce12b6fd153809a6e66047e4d3b, expected sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2: failed precondition
#31 exporting to image
#31 exporting layers 9.7s done
#31 CANCELED
------
> [linux/amd64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar:
------
docker-agent.Dockerfile:60
--------------------
58 |
59 |
60 | >>> COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
61 | RUN chmod 0644 /usr/share/jenkins/agent.jar \
62 | && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
--------------------
error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2": unexpected commit digest sha256:9bc7b10ff3793ea67fd1628262aafe7e24319ce12b6fd153809a6e66047e4d3b, expected sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2: failed precondition
second failure：
#30 [linux/arm64 stage-2 4/8] RUN DEBIAN_FRONTEND=noninteractive apt-get update && DEBIAN_FRONTEND=noninteractive apt-get --yes --no-install-recommends install ca-certificates curl fontconfig git git-lfs less netbase openssh-client patch tzdata && apt-get clean && rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/*
#30 CACHED
#31 [linux/arm64 stage-2 3/8] COPY --from=jdk-focal /opt/java/openjdk /opt/java/openjdk
#31 CACHED
#32 [linux/arm64 stage-2 2/8] RUN groupadd -g 1000 jenkins && useradd -d "/home/jenkins" -u 1000 -g 1000 -l -m -s /bin/bash jenkins
#32 CACHED
#33 [linux/arm64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
#33 ERROR: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6: failed precondition
------
> [linux/arm64 stage-2 5/8] COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar:
------
docker-agent.Dockerfile:60
--------------------
58 |
59 |
60 | >>> COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
61 | RUN chmod 0644 /usr/share/jenkins/agent.jar \
62 | && ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
--------------------
error: failed to solve: failed to compute cache key: failed commit on ref "layer-sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6": unexpected commit digest sha256:e85be1e84e11ec25c2492d934aff8a189811fd70377dbe19baf7b16a3aaba659, expected sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6: failed precondition
The third run build succeeded, I'm not sure if this has anything to do with the architecture I specified.
Buildx version
github.com/docker/buildx v0.8.2 6224def4dd2c3d347eee19db595348c50d7cb491
Docker info
No response
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
builder * kubernetes
amd64 kubernetes:///builder?deployment=amd64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/amd64*
arm64 kubernetes:///builder?deployment=arm64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/arm64*
Configuration
ARG VERSION
FROM jenkins/agent:3107.v665000b_51092-5-jdk11 as jdk11
FROM ubuntu:22
ENV JAVA_HOME=/opt/java/openjdk
ENV PATH "${JAVA_HOME}/bin:${PATH}"
...
RUN DEBIAN_FRONTEND=noninteractive apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get --yes --no-install-recommends install \
ca-certificates \
curl \
fontconfig \
git \
git-lfs \
less \
netbase \
openssh-client \
patch \
tzdata \
&& apt-get clean \
&& rm -rf /tmp/* /var/cache/* /usr/share/doc/* /usr/share/man/* /var/lib/apt/lists/*
COPY --from=jdk11 /usr/share/jenkins/agent.jar /usr/share/jenkins/agent.jar
RUN chmod 0644 /usr/share/jenkins/agent.jar \
&& ln -sf /usr/share/jenkins/agent.jar /usr/share/jenkins/slave.jar
...
Build logs
No response
Additional info
According to the description in this #235, I understand that multi-arch builds are executed in parallel.
I repeated the operation for the third time and succeeded. Checking the logs of the first two times shows that the expected digest is a certain layer in the image, and the results of the two error reports are not of the same architecture.
first expected: sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6 second expected: sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6
➜ docker buildx imagetools inspect docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11 | grep -B2 "linux/a"
Name: docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:a5948dcbf6af739b89c551b5eb96709e6f6d13aabd992a4cecf0c8b20ae13a1b
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/amd64
--
Name: docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:9ba35e957e4a2a39b1e9c7d2370eeb5c127269e71d78768ee3cb858f0bad2325
MediaType: application/vnd.oci.image.manifest.v1+json
Platform: linux/arm64
➜ oras manifest fetch docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:9ba35e957e4a2a39b1e9c7d2370eeb5c127269e71d78768ee3cb858f0bad2325
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:797ccab10de1965272376dcfd38fcb4fd45718a0964bb30095e5a636ecb133db",
"size": 7221
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:0f5fe16b1836feccd4765ac5685fc7a7b9c73445cac94fc595d2ffbc3cb59a7a",
"size": 53703215
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:40e542b51b4fa61d5fe9cf01aeca42c1cba52fe75e6c34ff9a318cd9b7004166",
"size": 3345
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:f050ca02276b14127d4f4a7853392e5c1a4029d31c8337436fc8d7a0dd015ce6",
"size": 36819598
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:aad4631e2640a7357148a952d7aa21fe29bc0f9ab0a3bec3ee774ac6f7234bbe",
"size": 1211879
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:94cd29ebaad3c8c6c159f1014d697704cfb18a1eebdcbfbf5413de99c5f459c6",
"size": 1211972
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:fc7932cf2fd9b57929c0452a37633740986a954029ce13e535bf88613918c964",
"size": 63444276
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:3899b98824724066ba5b1c916e0571acf76f3f427bdd0a484baed39569bac53b",
"size": 169
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
"size": 32
}
]
}%
➜ oras manifest fetch docker.io/jenkins/agent:3107.v665000b_51092-5-jdk11@sha256:a5948dcbf6af739b89c551b5eb96709e6f6d13aabd992a4cecf0c8b20ae13a1b
{
"mediaType": "application/vnd.oci.image.manifest.v1+json",
"schemaVersion": 2,
"config": {
"mediaType": "application/vnd.oci.image.config.v1+json",
"digest": "sha256:303477e0779f3b5451f41fc29bfc370be6d2f854439c33c5b7279016c90e893f",
"size": 7221
},
"layers": [
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:32fb02163b6bb519a30f909008e852354dae10bdfd6b34190dbdfe8f15403ea0",
"size": 55045922
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:18f33ebd1600b157b7fad942042fd1de0291734156602e239f71dcdb394a6059",
"size": 3339
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:a67f7b517bb981d7d75aa7c623acbdb669a12b61fb00b40c928aa9a55f26cea2",
"size": 37150529
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:c8e02e85f6f6109e22689e05f36dc9955b2e16fbb7e17b35c78e3d23e7e41be2",
"size": 1211874
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:6cb88d50449b2f3a875c53c37fd9d28d030cc949a77277cf8bb29b76e51b9a92",
"size": 1211970
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:52a4b0758312e424d845a32169faa3f54412a011e281c66e8f5ada72e357bd90",
"size": 65031229
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:fdafead6b594fcb15d4d5cb0e0b400b8573c4f60d2ca21b8353e716317835d18",
"size": 174
},
{
"mediaType": "application/vnd.oci.image.layer.v1.tar+gzip",
"digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
"size": 32
}
]
}%
Running into this too, except it's single platform but happening when using docker buildx bake for a multi-stage build with multiple targets
Sorry for the delay
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS builder * kubernetes amd64 kubernetes:///builder?deployment=amd64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/amd64* arm64 kubernetes:///builder?deployment=arm64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/arm64*
I see you're building using the kubernetes driver, does it happen only with this driver? And if possible can you give buildkit logs from the nodes?
Also can you check if you still repro with latest Buildx/BuildKit? Thanks!
Facing same issue, but only with some specific Dockerfile. Has anyone able to find any solution to this ?
Sorry for the delay
Builders list
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS builder * kubernetes amd64 kubernetes:///builder?deployment=amd64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/amd64* arm64 kubernetes:///builder?deployment=arm64&kubeconfig=/Users/mac/.kube/admin.conf inactive linux/arm64*I see you're building using the kubernetes driver, does it happen only with this driver? And if possible can you give buildkit logs from the nodes?
Also can you check if you still repro with latest Buildx/BuildKit? Thanks!
This issue has been around for a long time, and it's impossible to get information.
For some time now, I have been updating the latest buildx and haven't received any similar feedback.