cert-manager
Investigate & add an FAQ/warning about images rolled back after GitOps upgrade
See cert-manager#4207 for context- it seems like users have reported a similar issue a few times on Slack. It may be worth trying to reproduce it and adding a brief FAQ. Perhaps having a way to simply determine/verify the version of cert-manager installation would have help too.
Was anyone able to look into this and maybe provide a more detailed explanation of ths issue and how to resolve? We saw this on one cluster randomly so would like to understand the root cause. Thanks.
Hi @baileydoestech, no there haven't been any further developments in regards to this.
Perhaps you could add some details about your setup.
At the moment I am not sure whether this is a generic issue with soem GitOps tool or something cert-manager installation specific.
In our case the issue seemed to be some sort of race condition where ArgoCD (our Gitops tool) would successfully update the CRDs but fail to update the pods. We saw this in 2 out of 6 identical clusters.
An initial theory was that ArgoCD tries to update the ClusterIssuer using the old conversion webhook, hence the failure.
The fix was relatively striaghtforward (delete the cert-manager deployments to let ArgoCD resync) but I would be interested to know if this is what others saw (you mentioned Slack) and if our theory makes sense?
I'll close this issue since we do no longer have a conversion webhook. Please reopen the issue if it is still relevant.
