Mapped KubernetesPodOperater not rendering nested templates
Apache Airflow version
2.3.3
What happened
Nested values, such as env_vars for the KubernetesPodOperater are not being rendered when used as a dynamically mapped operator.
Assuming the following:
op = KubernetesPodOperater.partial(
env_vars=[k8s.V1EnvVar(name='AWS_ACCESS_KEY_ID', value='{{ var.value.aws_access_key_id }}')],
# Other arguments
).expand(arguments=[[1], [2]])
The Rendered Template results for env_vars should be:
("[{'name': 'AWS_ACCESS_KEY_ID', 'value': 'some-super-secret-value', 'value_from': None}]")
Instead the actual Rendered Template results for env_vars are un-rendered:
("[{'name': 'AWS_ACCESS_KEY_ID', 'value': '{{ var.value.aws_access_key_id }}', 'value_from': None}]")
This is probably caused by the fact that MappedOperator is not calling KubernetesPodOperater._render_nested_template_fields.
What you think should happen instead
No response
How to reproduce
No response
Operating System
Ubuntu 18.04
Versions of Apache Airflow Providers
No response
Deployment
Other 3rd-party Helm chart
Deployment details
No response
Anything else
No response
Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
MappedOperator inherits AbstractOperator, which calls _render_nested_template_fields via render_template. So the analysis seems off.
Does this work?
KubernetesPodOperater(
task_id=...,
...,
env_vars=[k8s.V1EnvVar(name='AWS_ACCESS_KEY_ID', value='{{ var.value.aws_access_key_id }}')],
)
Does this work?
KubernetesPodOperater( task_id=..., ..., env_vars=[k8s.V1EnvVar(name='AWS_ACCESS_KEY_ID', value='{{ var.value.aws_access_key_id }}')], )
Yes, as mentioned above the issue is only when using the operator as a mapped-operator.