docker manifest: allow to update an existing manifest with new images

[mweibel]

I'm using gitlab jobs to build multiple images independent of each other for different architectures (linux and several windows versions). My guess was that docker manifest create --amend {list} {image} would either create a new manifest list with the image passed or amend the existing list with the new image. e.g. my assumption was

# job 1
$ docker manifest create --amend cli:0.0.1 cli:0.0.1-linux
# job 2
$ docker manifest create --amend cli:0.0.1 cli:0.0.1-windows-1809
# job 3
$ docker manifest create --amend cli:0.0.1 cli:0.0.1-windows-1909

would result in a manifest list containing the three images for linux and the two windows images. This only works if you have the manifest locally already, however if you execute those three commands in different nodes or with clearing the docker directory in between, it would just always create a new manifest list containing one image.

Therefore I'd like a command which allows this, and/or a docker manifest pull command allowing me to retrieve the manifest list locally before updating it.

[thaJeztah]

If you have docker buildx installed, I think this is possible with the --apend option on docker buildx imagetools create; https://docs.docker.com/engine/reference/commandline/buildx_imagetools_create/

I must admit that I haven't used it myself, but perhaps @crazy-max or @tonistiigi can provide more information on that.

(if there's a full example, it would be good to add it to the docs as well https://github.com/docker/buildx/blob/master/docs/reference/buildx_imagetools_create.md)

[mweibel]

thanks! I have only briefly looked at buildx because of two reasons:

• I assumed docker manifest would be enough to work with
• buildx seems to work together with buildkit which in turn doesn't work with windows containers

Unsure if I can use it in this case, but I can try :) Anyhow, I still wonder what should be buildx and what manifest, but I think the manifest create --amend works in a weird and not expected way, to me at least.

[Koubek]

I fully agree with @mweibel. docker manifest IMHO doesn't provide full functionality out of the box. Are there any plans to improve this command or is it something that is basically dead and won't be updated at all?

[tushar-door]

@mweibel : were you able to workaround this? I was thinking if we can push a manifest and for next edit,

• inspect existing
• recreate a new one using inspect info from existing and new image addition
• re-push same version manifest to replace old.

thoughts?

[mweibel]

I'm just using one docker manifest create with all image tags in one command at the moment and restructured the build pipeline around this.

[pfuntner]

I'm also very interested in being able to update an existing manifest list and replace just one image. We're an Ansible shop and are looking at creating amd64/arm64 Linux images separately. Instead of using docker build or docker buildx, we use packer build which applies our Ansible playbook.

Update

I was able to make use of docker manifest inspect to update a manifest list and replace a single image, retaining an existing image. We have a manifest list on a registry with two architectures:

$ docker manifest inspect REPO:latest | jq '.manifests[] | { "arch": .platform.architecture, "digest": .digest }'
{
  "arch": "amd64",
  "digest": "sha256:590d1e9deea7b7ec17c4cdade7aca30b3db400603158c59d8402c7a79f048258"
}
{
  "arch": "arm64",
  "digest": "sha256:531453a6145ff2768f85f141ad0c8a404889ea2af5a7be922d8e6f3ca36e9fa8"
}

To update manifest list with a new amd64 image:

1. Push image new tag:

docker push REPO:2022-06-30_11:28

2. Create/push manifest list with new image and old arm64:

$ docker mainfest create REPO:latest REPO:2022-06-30_11:28 REPO@$(docker manifest inspect REPO:latest | jq '.manifests[] | select(.platform.architecture != "amd64") | .digest' | tr -d \")
$ docker mainfest push REPO:latest