airflow icon indicating copy to clipboard operation
airflow copied to clipboard

KubernetesPodOperator rendered template tab does not pretty print `env_vars`

Open ShahNewazKhan opened this issue 3 years ago • 6 comments

Apache Airflow version

2.2.5

What happened

I am using the KubernetesPodOperator for airflow tasks in Airflow 2.2.5 and it doesnot render the env_vars in the rendered template in a easily human consumable format as it did in Airflow 1.10.x

image

What you think should happen instead

The env_vars should be pretty printed in human legible form.

How to reproduce

Create a task with the KubernetesPodOperator and check the Rendered template tab of the task instance.

Operating System

Docker

Versions of Apache Airflow Providers

2.2.5

Deployment

Other Docker-based deployment

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

ShahNewazKhan avatar Jun 04 '22 20:06 ShahNewazKhan

Thanks for opening your first issue here! Be sure to follow the issue template!

boring-cyborg[bot] avatar Jun 04 '22 20:06 boring-cyborg[bot]

@ShahNewazKhan WDYT about submitting a PR for this? Airflow has over 2K contributors and it seems like you are in a great position to test an enhancement here.

I suspect it's a matter of adding template_fields_renderers attribute to KubernetesPodOperator; maybe even something similar to what's done in this operator (and others).

There are several rendering lexers that are available and finding the correct one should make env_vars more human-readable in the UI.

josh-fell avatar Jun 06 '22 15:06 josh-fell

I took a crack at this but I don't think it is easily possible to make the env_vars formatting nicer. I tried adding this line to the k8s pod operator template_fields_renderers = {"env_vars": "json"}

The env_vars look like this in the UI afterwards: image

In the past it looks like env_vars was just a dict so it was easy to format it with the json lexer however now env_vars are converted to a list of env_vars with this function.

Perhaps there is a way to still make this work, but it isn't as straight forward as I was hoping.

DMilmont avatar Jun 10 '22 16:06 DMilmont

Thanks for looking into this @DMilmont!

@josh-fell unfortunately I do not have the bandwidth to take this on, just reporting here, for the time being I will use the human readable env_vars dict passed in from the task definition.

ShahNewazKhan avatar Jun 10 '22 17:06 ShahNewazKhan

Changing the render to use python solves the issue:

template_fields_renderers = {"env_vars": "py"}

bdsoha avatar Aug 07 '22 12:08 bdsoha

Changing the render to use python solves the issue:

template_fields_renderers = {"env_vars": "py"}

Feel free to submit PR.

potiuk avatar Aug 07 '22 13:08 potiuk

I will make a PR for this! Feel free to assign.

DMilmont avatar Aug 20 '22 19:08 DMilmont