kubernetes-sigs
Kind node image does not ship containerd v2.1.4
What happened:
When creating a cluster using kind v0.3.0 the resulting nodes contain an unexpected containerd version - v2.1.3.
What you expected to happen:
containerd version in node is v2.1.4 as documented in the kind v0.3.0 release notes. containerd v2.1.4 addresses multiple issues, including one that impacts the ability of a node to pull images at all (https://github.com/containerd/containerd/pull/12127).
How to reproduce it (as minimally and precisely as possible):
Create a cluster with kind v0.3.0. This affects all clusters equally.
Anything else we need to know?:
Some digging showed me that the base image used by kind to build node images has not been updated with the v0.3.0 release. This includes both the image itself, which has not been updated, as well as the reference to the image (pkg/build/nodeimage/defaults.go).
When building the base image using the tooling in the repository one ends up with an image that comes with containerd v2.1.4, but still contains v0.29.0 in the image tag. This in turn seems to be caused by v0.3.0 being a lightweight git tag and not an annotated git tag.
Environment:
- kind version: (use
kind version): kind v0.30.0 go1.25.0 darwin/arm64
There is also this kubernetes 1.34.0 CRD bug that got fixed in 1.34.1 https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.34.md#changelog-since-v1340
It seems containerd is targeting 2.2.0 for nov 5 https://github.com/containerd/containerd/milestones
So, we can do a new patch and fix current problem and update to latest the dependencies
@stmcginnis @BenTheElder WDYT
We are expecting a 1.34.2 next week from Kubernetes as well to consider.
I'm not sure if we should wait for 2.2 or go ahead with patching to have 2.1.4, but it might make sense to pick up 1.34.2 as the default because 1.34.x has had some noteworthy fixes and kinv v0.30 is defaulting to 1.34.0
Create a cluster with kind v0.3.0. This affects all clusters equally.
Some digging showed me that the base image used by kind to build node images has not been updated with the v0.3.0 release.
Aside, these should say "v0.30.0", right? Unless they are the release from 2019 (doesn't seem so)
When building the base image using the tooling in the repository one ends up with an image that comes with containerd v2.1.4, but still contains v0.29.0 in the image tag. This in turn seems to be caused by v0.3.0 being a lightweight git tag and not an annotated git tag.
It's not related to the git tagging, it's due to the base image being a pre-built artifact that we don't live build for every node image build, it was not updated properly or the release notes are in error.
Aside, these should say "v0.30.0", right? Unless they are the release from 2019 (doesn't seem so)
You are absolutely right, typo on my end.
It's not related to the git tagging, it's due to the base image being a pre-built artifact that we don't live build for every node image build, it was not updated properly or the release notes are in error.
This is true, I wanted to share the finding that - independent from the missing base image update - git describe still reports back v0.29.0 as this was the last annotated tag. Just in case you run into this situation as well.
Thanks. I don't think this is related, but we should revisit if the release scripting should prefer annotated tags or not.
I intended to make sure we did a patch release following the Kubernetes patch releases so we can batch this up with 1.34.2 bug fixes, I know of at least one that would be good to have available before kubecon, but those were delayed again it seems.
Previously expected one week delay to the 22nd: https://groups.google.com/a/kubernetes.io/g/dev/c/pfJfn0YUeqo/m/DQ-FIaU4BAAJ
Following up with SIG Release about that first.
sorry to bother, will there be an updated Docker hub kindest/base image with a suitable tag? The latest "stable" is v20250521-31a79fd4. I understand that this image is not intended for reuse, yet would like to leverage the updated containerd and the image in my CRI event-pleg API tests in a different opensource project.
We will be doing a new release very soon, though no specific date has been set yet.
We have https://github.com/kubernetes-sigs/kind/pull/4056 pending, I've been out a bit more recently and the Kubernetes (main repo) patch releases were delayed previously, we'll probably try to put out a release aligned with k8s 1.35
I understand that this image is not intended for reuse, yet would like to leverage the updated containerd and the image in my CRI event-pleg API tests in a different opensource project.
Yes, you can do this, but it's not supported, and we're not really building the dev cycle around it. You could build the image yourself if you're using the base image directly to do other things. See images/base, but please read the docs around expectations first. We do not support depending on the contents of this image and they are subject to change at any time.