Post-renderer not receiving chart hook manifests?

[robg]

Output of helm version:

version.BuildInfo{Version:"v3.1.2", GitCommit:"d878d4d45863e42fd5cff6743294a11d28a9abce", GitTreeState:"clean", GoVersion:"go1.13.8"}

Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4+k3s1", GitCommit:"3eee8ac3a1cf0a216c8a660571329d4bda3bdf77", GitTreeState:"clean", BuildDate:"2020-03-25T16:13:25Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4+k3s1", GitCommit:"3eee8ac3a1cf0a216c8a660571329d4bda3bdf77", GitTreeState:"clean", BuildDate:"2020-03-25T16:13:25Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}

I was experimenting with the --post-renderer option and a simple chart that includes one pod:

apiVersion: v1
kind: Pod
metadata:
  name: post-render-test
  labels:
    # The "app.kubernetes.io/managed-by" label is used to track which tool
    # deployed a given chart. It is useful for admins who want to see what
    # releases a particular tool is responsible for.
    app.kubernetes.io/managed-by: {{.Release.Service | quote }}
    # The "app.kubernetes.io/instance" convention makes it easy to tie a release
    # to all of the Kubernetes resources that were created as part of that
    # release.
    app.kubernetes.io/instance: {{.Release.Name | quote }}
    app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
    # This makes it easy to audit chart usage.
    helm.sh/chart: "{{.Chart.Name}}-{{.Chart.Version}}"
    values: {{.Values.Name}}
spec:
  # This shows how to use a simple value. This will look for a passed-in value
  # called restartPolicy. If it is not found, it will use the default value.
  # {{default "Never" .restartPolicy}} is a slightly optimized version of the
  # more conventional syntax: {{.restartPolicy | default "Never"}}
  restartPolicy: {{default "Never" .Values.restartPolicy}}
  containers:
  - name: waiter
    image: "alpine:3.4"
    command: ["/bin/sleep","9000"]

and one pre-install hook:

apiVersion: batch/v1
kind: Job
metadata:
  name: "{{ .Release.Name }}"
  labels:
    app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
    app.kubernetes.io/instance: {{ .Release.Name | quote }}
    app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
    helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
  annotations:
    # This is what defines this resource as a hook. Without this line, the
    # job is considered part of the release.
    "helm.sh/hook": pre-install,pre-upgrade
    "helm.sh/hook-weight": "-5"
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  backoffLimit: 3 # retry this task 3 times
  template:
    metadata:
      name: "{{ .Release.Name }}"
      labels:
        app.kubernetes.io/managed-by: {{ .Release.Service | quote }}
        app.kubernetes.io/instance: {{ .Release.Name | quote }}
        helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
    spec:
      restartPolicy: Never
      containers:
      - name: pre-deploy-job
        image: "alpine:3.4"
{{- if .Values.predeploy_task.fail }}
        command: ["/bin/false"]
{{- else }}
        command: ["/bin/sleep","{{ default "10" .Values.sleepyTime }}"]
{{- end }}

For the test, I was using a no-op post-renderer:

#!/bin/bash

tee --append  <&0 all.yaml

When I execute helm install post-render-test ../chart --post-renderer ./post-renderer --debug --dry-run I see both the chart hook and the pod manifest in the output, however my post-renderer only captures the pod manifest.

I've tried this as well with helm template with the same results.

Is this the intended behavior? I couldn't find anything in the documentation to suggest that hooks would not be included when invoking the post-renderer.

[hickeyma]

@thomastaylor312 if you get a chance ^^

[thomastaylor312]

@robg You are correct. I took a look at the code and it does not pass hooks, just the main manifests. I don't think hooks were intended as part of the initial post render work, but neither were they specifically excluded. So I am going to label this as a bug for now.

If someone wants to take this on it shouldn't be too difficult. It won't be elegant, but it shouldn't be too bad. In renderResources, you'll need to add the hooks to the buffer sent in to the post renderer. The clunky part will come in splitting the manifests back out into "Files" to be sorted by the SortManifests function, which expects a map of filenames to strings containing content.

[dellkeji]

I have the same question.

i want to inject some label by post-renderer, but not work.

[technosophos]

I'm optimistically adding this to 3.3 in hopes that someone takes it up.

[kabakaev]

@robg, could you please test if the #7948 fixes the issue for you?

git clone https://github.com/kabakaev/helm.git
git checkout hook-post-renderer
make build-cross
ls _dist/*/helm

It worked for me with `helm install releasename chartname --post-renderer 'kubectl kustomize /some/directory'.

[dellkeji]

@robg, could you please test if the #7948 fixes the issue for you?

git clone https://github.com/kabakaev/helm.git
git checkout hook-post-renderer
make build-cross
ls _dist/*/helm

It worked for me with `helm install releasename chartname --post-renderer 'kubectl kustomize /some/directory'.

I buid helm bin from your branch, and use post-renderer with ytt; but not work.

Error: cannot prepare a file for post-process: document has no metadata:

use master branch, build helm bin is work.

[kabakaev]

@dellkeji, thanks for testing it!

Error: cannot prepare a file for post-process: document has no metadata:

There seem to be an empty template on the input. Do you build some public helm chart, so that I can also test it?

[kabakaev]

Ok, I think I know what happened. I forgot to check if the ‘document’ variable is not empty, which may happen with ‘Split()’. I’ll push the fix in an hour.

[kabakaev]

@dellkeji, how about now with the new commit?

git fetch
git checkout d9e44ad9793e8e3d639416ff9227dffbde768086
make build-cross

[robg]

@kabakaev Thanks for putting this patch together, sorry I missed your earlier message.

I just tried d9844ad and received the same error as @dellkeji:

(aws-cli) robg@robgnuc:~/projects/helm-poc/post-render/overlay(master)$ /home/robg/projects/patches/helm/_dist/linux-amd64/helm install post-render-test ../chart/ --post-renderer ./kustomize  --debug --dry-run
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /home/robg/projects/helm-poc/post-render/chart

Error: cannot prepare a file for post-process: document has no metadata:
helm.go:84: [debug] cannot prepare a file for post-process: document has no metadata:

FWIW, this error appears to happen after the post-renderer has been invoked.

[dellkeji]

@dellkeji, how about now with the new commit?

git fetch
git checkout d9e44ad9793e8e3d639416ff9227dffbde768086
make build-cross

@kabakaev thanks!

this commit branch is ok, passed the test of our case.

helm template ttt exam-chart -n default --post-renderer test-ytt/my_test

[kabakaev]

I just tried d9844ad and received the same error

@robg, please try 9c06aff.

[robg]

I just tried d9844ad and received the same error

@robg, please try 9c06aff.

@kabakaev that one worked for me. Thanks! I should say that I actually tested 1d30db8...

[bacongobbler]

3.3.0-rc.1 is being cut tomorrow, so we're removing this from the milestone.

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[nikals99]

Any update on when this fix will available ?

[will-beta]

expecting updates

[kschu91]

Any updates on this one?

[jycamier]

I need updates too.

In my use case with Kustomize, a ConfigMapGenerator creates some ConfigMaps which contain stuff for database migration, which is executed in a Job on pre-install and pre-upgrade hooks, so I cannot use the native way to propagate my name suffix (which is finally a way to patch but with a ConfigMapGenerator).

https://kubectl.docs.kubernetes.io/references/kustomize/configmapgenerator/#propagating-the-name-suffix

[KR411-prog]

We are facing same issue.. We have a job in Contour namely cert-gen. This has some pre-install hook annotations..However when we run this in post-renderer, it is not picking up this job. We just workaround this using helmsman hooks..However it will be good if this issue gets fixed.

[evanlwj]

Same issue happened to us in attempting to deploy helm package to airgap cloud, we need post-render to update the proper ACR in helm image path, any plan/eta to merge following fix? Thanks

[WIP] fix: send hook yaml into post-renderer (#7891) #7948

[stealthybox]

This is probably also applicable to helm test

[github-actions[bot]]

This issue has been marked as stale because it has been open for 90 days with no activity. This thread will be automatically closed in 30 days if no further activity occurs.

[mmslkr]

still relevant

[ThomasSteinbach]

Not stale, still relevant.

[dgaffuri]

Any plan to include fix in future releases?

[Chili-Man]

We've been bashing our heads around this behavior and just found out about this; at the minimum it would be great if this behavior was documented

[pavel-mitrofanov-regtech]

This issue is relevant for me since I wanted to use Helm post-renderer in order to overcome the issue of deploying a chart (GitLab but could any other) into an Istio-enabled namespace. Istio has a known issue with Jobs not completing: https://github.com/istio/istio/issues/11045

And the GitLab Helm chart uses a Job to initialize shared Secrets. Unfortunately the Job is created through a Helm hook, so it is not customizable using the post-renderer, due to this issue.

[sethgupton-mastery]

Just stumbled face first into this too. Hooks running through the postrenderer would be great.

[gaborho]

We are hindered by this limitation as well. Would be great if hook manifests would be passed over to post renderer.