airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Evaluate None in SQLAlchemy's extended JSON type decorator

Open shahar1 opened this issue 1 year ago • 0 comments

closes: #22245

As of today, at least in mapped operators for map_index > 0, the next_kwargs column is saved as the string null in the DB, instead of a real <null>. That is unlike mapped operators with indeces -1 or 0, as well as unmapped operators. As far as I'm concerned, this inconsistency doesn't affect any logic that depends on it, since it is still deserialized as Python's None - but it will be nice to save a few bytes and ensure consistency. While I haven't figured out why it happens only in that case - I've found that applying the flag should_evaluate_none = True seems to solve it. I didn't find a proper way to test it, since it is always deserialized as Python's None.

Mapped operators are yet to be implemented in the main with the new task sdk, but if tests won't indicate that it breaks anything - I think that it will be ok to merge.


^ Add meaningful description above Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

shahar1 avatar Dec 20 '24 18:12 shahar1