Incompatibility Issue with controller-runtime Version Pinning and envtest Setup - It is requiring go 1.22 when the latest release still using go 1.21 as old project versions

[camilamacedo86]

Problem Description

We've encountered an issue where our environment cannot pin a specific version of controller-runtime, leading to compatibility issues with earlier versions of kubebuilder when setting up envtest. This incompatibility prevents our projects from maintaining a consistent development environment, particularly affecting our CI/CD pipelines.

Error Output

While attempting to set up envtest (go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest), we're met with the following error message due to the version incompatibility besides the LATEST v0.17.2 does not support go 1.22 and old projects not be using it yet as well.

go: downloading sigs.k8s.io/controller-runtime/tools/setup-envtest v0.0.0-20240327193027-21368602d84b
go: downloading sigs.k8s.io/controller-runtime v0.17.2
go: sigs.k8s.io/controller-runtime/tools/setup-envtest@latest: sigs.k8s.io/controller-runtime/tools/[email protected] requires go >= 1.22.0 (running go 1.21.8; GOTOOLCHAIN=local)

This error highlights the requirement for Go version >= 1.22.0, while we are currently using Go 1.21.8.

Ideal Solution (controller-runtime provide a envtest setup solution that allows us pin the tag releases)

Ideally, we need a way to setup envtest that allows version pinning per tag release. This would enable our community to manage dependencies more effectively and ensure compatibility across different versions of kubebuilder and controller-runtime. Note that in many scenarios we might need to do this setup in shell scripts.

Alternative Solution (how we could fix it and which seems the best available approach so far)

Use the release branch, example : go install sigs.k8s.io/controller-runtime/tools/setup-envtest@release-17

[camilamacedo86]

@joelanford @varshaprasad96 @vincepri @sbueringer

I'm reaching out to gather insights or possible solutions, as we're currently facing issues with the existing setup. Our immediate goal is to enhance the experience for Kubebuilder users and resolve CI failures by utilizing the release branch of controller-runtime.

Here's the snippet from our current approach:

ENVTEST ?= $(LOCALBIN)/setup-envtest-$(ENVTEST_VERSION)
ENVTEST_VERSION ?= release-0.17
...

.PHONY: envtest
envtest: $(ENVTEST) ## Download setup-envtest locally if necessary.
$(ENVTEST): $(LOCALBIN)
	$(call go-install-tool,$(ENVTEST),sigs.k8s.io/controller-runtime/tools/setup-envtest,$(ENVTEST_VERSION))

While this method serves as a temporary fix, a more ideal solution would involve pinning specific tag versions to ensure stability and predictability. The reliance on the release branch introduces a risk, especially if a release branch is not created as part of the controller-runtime's release process. This potential inconsistency could adversely affect Kubebuilder and the broader community.

[lukas016]

proper versioning of setup-env is solving here https://github.com/kubernetes-sigs/controller-runtime/issues/2646 but maintainers don't react.

[sbueringer]

Hey folks, we are currently experimenting a bit on setup-envtest because the envtest binaries are today hosted on the Google-owned kubebuilder repository.

Going forward we will start hosting the envtest binaries on controller-tools releases. We currently also plan to move setup-envtest to controller-tools as it fits better there. As part of that we will build & publish the setup-envtest binary on controller-tools releases in the future. This should also give proper versioning. https://github.com/kubernetes-sigs/controller-runtime/issues/2646#issuecomment-2066758692

Let me know if this covers the issue.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[camilamacedo86]

Hi @sbueringer,

Issue comment

Thank you for your input on this issue. However, the challenge we're facing isn't fully addressed by the current solution.

The problem

The core issue relates to the versioning of setup-envtest itself. Specifically, we are unable to run:

go install sigs.k8s.io/controller-runtime/tools/setup-envtest@<controller-runtime-tag-release>

This means we can't align the setup-envtest version precisely with the controller-runtime version updates. We faced some incompatible scenarios with the latest.

Solution (Best approach find so far)

Then, to circumvent potential problems, we are currently scaffolding kubebuilder with:

go install sigs.k8s.io/controller-runtime/tools/setup-envtest@<branch-release>
// We can only use a branch release OR latest. 

The downside of using the branch is if a release is made from master without updating/creating the branch, it could lead to issues, although so far, this method has helped us avoid any major problems.

Conclusion

However, I think we can recommend the branch release as a workaround for these limitations as a satisfactory solution. In this way, since we've switched to using the branch release, and we haven't encountered significant issues and I'm unsure how if we have a technical solution to could effectively tag and release envtest-setup to match controller-runtime. Therefore, I am comfortable with closing the issue on those terms.

So I will close it based on the above, if anyone does not agree, they can open a new issue pointing to this one and propose a better solution.

Thank you for considering this perspective.

[sbueringer]

As part of that we will build & publish the setup-envtest binary on controller-tools releases in the future. This should also give proper versioning.

This plan slightly changed (at least in my head). I would now publish setup-envtest binaries on controller-runtime releases. So my question was something like: If every controller-runtime release has attachments with setup-envtest binaries for various OS / ARCH, would this address your problem? (the binary would have to be downloaded instead of compiled locally, but that seems more of a benefit to me)

[sbueringer]

Please see https://github.com/kubernetes-sigs/controller-runtime/pull/2911 With this PR, setup-envtest binaries for every upcoming controller-runtime release can simply be downloaded from release attachments

[sbueringer]

Now starting with the v0.19 release setup-envtest binaries are available via release attachments (see e.g. https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.19.0)

I would recommend using these binaries if:

• a specific setup-envtest version is required
• folks have trouble compiling the binary themselves