Changing TargetNamespace Causes Rollback Failure on Update

[jonathan-innis]

When there is a change to the spec.targetNamespace is changed in the application group, the reconciler is unable to rollback the previous workflow as it can't find the previous one

2021-04-10T19:47:29.611Z        ERROR   controllers.ApplicationGroup    failed to rollback failed HelmRelease instances {"appgroup": "bookinfo", "error": "release: not found", "errorVerbose": "release: not found\nhelm.sh/helm/v3/pkg/storage/driver.init\n\t/go/pkg/mod/helm.sh/helm/[email protected]/pkg/storage/driver/driver.go:27\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5228\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:190\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"}
2021-04-10T19:47:29.611Z        ERROR   controller-runtime.controller   Reconciler error        {"controller": "applicationgroup", "request": "/bookinfo", "error": "release: not found", "errorVerbose": "release: not found\nhelm.sh/helm/v3/pkg/storage/driver.init\n\t/go/pkg/mod/helm.sh/helm/[email protected]/pkg/storage/driver/driver.go:27\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5228\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:190\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"}
2021-04-10T19:47:34.733Z        ERROR   controllers.ApplicationGroup            {"appgroup": "bookinfo", "error": "workflow in failure/error condition : helmrelease in failure status"}
2021-04-10T19:47:34.768Z        INFO    controllers.ApplicationGroup    Remediating the applicationgroup with helmrelease failure status        {"appgroup": "bookinfo"}
2021-04-10T19:47:34.768Z        DEBUG   controller-runtime.manager.events       Warning {"object": {"kind":"ApplicationGroup","name":"bookinfo","uid":"9d8f0d37-d60a-49c3-817d-3039c545ca3b","apiVersion":"orkestra.azure.microsoft.com/v1alpha1","resourceVersion":"2260"}, "reason": "ReconcileError", "message": "Failed to reconcile ApplicationGroup bookinfo with Error : workflow in failure/error condition : helmrelease in failure status"}
2021-04-10T19:47:34.777Z        ERROR   controllers.ApplicationGroup    failed to rollback failed HelmRelease instances {"appgroup": "bookinfo", "error": "release: not found", "errorVerbose": "release: not found\nhelm.sh/helm/v3/pkg/storage/driver.init\n\t/go/pkg/mod/helm.sh/helm/[email protected]/pkg/storage/driver/driver.go:27\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5228\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.doInit\n\t/usr/local/go/src/runtime/proc.go:5223\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:190\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1357"}

[nitishm]

I doubt changing targetNamespaces is supported by helm itself. Since helm releases are namespaced and locked to the namespace that they are deployed to, it might not be possible to move them to a new namespace.

[nitishm]

This is an old issue but I suppose it holds true for helmv3 as well - https://github.com/helm/helm/issues/3657

[nitishm]

We will eventually need Validating webhooks to prevent users from upgrading these fields.

[nitishm]

@jonathan-innis Can we retry this with the migration to helm-controller?

[jonathan-innis]

Haven't tried this, need to track this as an item to attempt