buildx
buildx copied to clipboard
docker
Cache miss when pushing only 1 target
There seems to be a cache miss when you build all targets but only push one target to a repository. Please find the relevant logs and config below.
I am having the same issue. Two consecutive runs do have a cache miss
Docker version
ubuntu@ip-172-31-88-222:~$ docker --version
Docker version 20.10.5, build 55c4c88
bake.hcl
variable "CT_ECR_PREFIX" {
default = ""
}
variable "CT_DOCKER_CACHE" {
default = ""
}
variable "GITHUB_SHA" {
}
variable "DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME" {
}
variable "CT_ECR_RELEASE_TAG" {
}
group "default" {
targets = ["app", "fake_aws"]
}
target "cache" {
cache-to = ["type=inline,mode=max"]
cache-from = [
"${CT_ECR_PREFIX}ct-base:${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
"${CT_ECR_PREFIX}ct-base:latest",
"${CT_ECR_PREFIX}ct-base:ct-addon_${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
"${CT_ECR_PREFIX}ct-base:ct-addon_latest"
]
}
target "app" {
inherits = ["cache"]
args = {
CT_DOCKER_CACHE = "${CT_DOCKER_CACHE}"
CT_ECR_PREFIX = "${CT_ECR_PREFIX}"
GITHUB_SHA = "${GITHUB_SHA}"
}
tags = [
"${CT_ECR_PREFIX}ct-base:ct-addon_${GITHUB_SHA}",
"${CT_ECR_PREFIX}ct-base:ct-addon_${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
"${CT_ECR_PREFIX}ct-base:ct-addon_${CT_ECR_RELEASE_TAG}"
]
}
target "fake_aws" {
inherits = ["cache"]
context = "../../../tools/mock_servers/fake_aws"
args = {
CT_DOCKER_CACHE = "${CT_DOCKER_CACHE}"
CT_ECR_PREFIX = "${CT_ECR_PREFIX}"
GITHUB_SHA = "${GITHUB_SHA}"
}
tags = [
"ct-fake-aws:${GITHUB_SHA}",
"ct-fake-aws:${DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME}",
"ct-fake-aws:${CT_ECR_RELEASE_TAG}"
]
}
Build without push
ubuntu@ip-172-31-88-222:~/workspace/foo_ct-backend_PR-10594/js/service/addon$ CI=true CT_DOCKER_CACHE=harbor.infra.foo.com/cache/ CT_ECR_PREFIX=01234567890.dkr.ecr.eu-west-1.amazonaws.com/ DOCKER_CLI_EXPERIMENTAL=enabled GITHUB_SHA=a4e979fac131a5b8c39d4e2f0e8b4f745730941f DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME=chore_core-buildkit CT_ECR_RELEASE_TAG=chore_core-buildkit docker buildx bake -f bake.hcl
[+] Building 3.5s (21/21) FINISHED
=> [app internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [fake_aws internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 243B 0.0s
=> [app internal] load .dockerignore 0.1s
=> => transferring context: 35B 0.0s
=> [fake_aws internal] load .dockerignore 0.1s
=> => transferring context: 109B 0.0s
=> [fake_aws internal] load metadata for 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com 0.0s
=> [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:chore_core-buildkit 0.0s
=> [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:latest 0.2s
=> [fake_aws] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit 0.0s
=> [app] importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_latest 3.2s
=> [fake_aws 1/4] FROM 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> [app internal] load build context 0.0s
=> => transferring context: 10.16kB 0.0s
=> [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com 0.0s
=> [fake_aws internal] load build context 0.1s
=> => transferring context: 4.61kB 0.0s
=> CACHED [app 2/4] WORKDIR /home/ct/js/service/addon 0.0s
=> CACHED [app 3/4] COPY . . 0.0s
=> CACHED [app 4/4] RUN yarn build 0.0s
=> CACHED [fake_aws 2/3] WORKDIR /home/ct/tools/mock_servers/fake_aws 0.0s
=> CACHED [fake_aws 3/3] COPY . /home/ct/tools/mock_servers/fake_aws 0.0s
=> [fake_aws] exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8 0.0s
=> => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit 0.0s
=> => writing image sha256:d770cc800c787c7823debc6249cf26904b26c0e3f90ae00f7121c495ae909918 0.0s
=> => naming to docker.io/library/ct-fake-aws:a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> => naming to docker.io/library/ct-fake-aws:chore_core-buildkit 0.0s
=> [fake_aws] exporting cache 0.0s
=> => preparing build cache for export 0.0s
Docker image hash
ubuntu@ip-172-31-88-222:~$ docker inspect 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f | jq -r '.[].Id'
sha256:8fbb5dd0bff70b92372826d966cfb7c3452cfb89bccd2a9b484b2e4bfabc0851
Build with push
ubuntu@ip-172-31-88-222:~/workspace/foo_ct-backend_PR-10594/js/service/addon$ CI=true CT_DOCKER_CACHE=harbor.infra.foo.com/cache/ CT_ECR_PREFIX=01234567890.dkr.ecr.eu-west-1.amazonaws.com/ DOCKER_CLI_EXPERIMENTAL=enabled GITHUB_SHA=a4e979fac131a5b8c39d4e2f0e8b4f745730941f DOCKERIZED_FRIENDLY_GIT_BRANCH_NAME=chore_core-buildkit CT_ECR_RELEASE_TAG=chore_core-buildkit docker buildx bake -f bake.hcl --push app
[+] Building 1.2s (17/17) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 208B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 268B 0.0s
=> [internal] load metadata for 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:latest 0.2s
=> importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit 0.0s
=> importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:chore_core-buildkit 0.0s
=> importing cache manifest from 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_latest 0.2s
=> [1/4] FROM 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 8.92MB 0.1s
=> [auth] sharing credentials for 01234567890.dkr.ecr.eu-west-1.amazonaws.com 0.0s
=> CACHED [2/4] WORKDIR /home/ct/js/service/addon 0.0s
=> CACHED [3/4] COPY . . 0.0s
=> CACHED [4/4] RUN yarn build 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8 0.0s
=> => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f 0.0s
=> => naming to 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit 0.0s
=> exporting cache 0.0s
=> => preparing build cache for export 0.0s
=> pushing 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f with docker 0.4s
=> => pushing layer e78ea794e900 0.5s
=> => pushing layer 04203156884c 0.5s
=> => pushing layer 5f70bf18a086 0.5s
=> => pushing layer c29abcf4219d 0.5s
=> => pushing layer 2c9612efc5d6 0.5s
=> => pushing layer 3109d88adfae 0.5s
=> => pushing layer de48e899da14 0.5s
=> => pushing layer ac6f92babc19 0.5s
=> => pushing layer 4032548fa860 0.5s
=> => pushing layer 03dbc70c199c 0.5s
=> => pushing layer f806f4073cf5 0.5s
=> => pushing layer 462c78d623f8 0.5s
=> => pushing layer c0f63f893bfb 0.5s
=> => pushing layer 987d9826e606 0.5s
=> => pushing layer d2dc08645627 0.5s
=> => pushing layer 16bf97960ee7 0.5s
=> => pushing layer 190990e6fae4 0.5s
=> => pushing layer e44afd48b6d5 0.5s
=> => pushing layer 799e7111d6d4 0.5s
=> => pushing layer a72a7e555fe1 0.5s
=> => pushing layer b8f8aeff56a8 0.5s
=> => pushing layer 687890749166 0.5s
=> => pushing layer 2f77733e9824 0.5s
=> => pushing layer 97041f29baff 0.5s
=> pushing 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_chore_core-buildkit with docker 0.4s
=> => pushing layer e78ea794e900 0.4s
=> => pushing layer 04203156884c 0.4s
=> => pushing layer 5f70bf18a086 0.4s
=> => pushing layer c29abcf4219d 0.4s
=> => pushing layer 2c9612efc5d6 0.4s
=> => pushing layer 3109d88adfae 0.5s
=> => pushing layer de48e899da14 0.5s
=> => pushing layer ac6f92babc19 0.5s
=> => pushing layer 4032548fa860 0.5s
=> => pushing layer 03dbc70c199c 0.5s
=> => pushing layer f806f4073cf5 0.5s
=> => pushing layer 462c78d623f8 0.5s
=> => pushing layer c0f63f893bfb 0.5s
=> => pushing layer 987d9826e606 0.5s
=> => pushing layer d2dc08645627 0.5s
=> => pushing layer 16bf97960ee7 0.5s
=> => pushing layer 190990e6fae4 0.5s
=> => pushing layer e44afd48b6d5 0.5s
=> => pushing layer 799e7111d6d4 0.5s
=> => pushing layer a72a7e555fe1 0.5s
=> => pushing layer b8f8aeff56a8 0.5s
=> => pushing layer 687890749166 0.5s
=> => pushing layer 2f77733e9824 0.5s
=> => pushing layer 97041f29baff 0.0s
Docker image hash
ubuntu@ip-172-31-88-222:~$ docker inspect 01234567890.dkr.ecr.eu-west-1.amazonaws.com/ct-base:ct-addon_a4e979fac131a5b8c39d4e2f0e8b4f745730941f | jq -r '.[].Id'
sha256:b762d9b76d272ac22305cbcbefef2a28ecb6d70baffecd008e75913d0620ddc8
I am not sure if the cache miss happens when only pushing one target or cache miss happens when pushing irrespective of the targets