marquez icon indicating copy to clipboard operation
marquez copied to clipboard

Add parentRun alias to LineageEvent RunFacet to support older OpenLin…

Open collado-mike opened this issue 3 years ago • 0 comments

Problem

Prior to https://github.com/OpenLineage/OpenLineage/pull/1037 , all Airflow OpenLineage events reported their parent runs as parentRun rather than the spec-defined parent facet. This change adds an alias to support those older events.

Unfortunately, jackson seems to prefer the alias field name rather than the target (or maybe it's ordered on some condition I don't understand), so if an event defines both, the parentRun facet will be used. This is confirmed in a unit test. The OpenLineage integration does use the same value for both parentRun and parent facets, so this should not cause any issues.

Note: All database schema changes require discussion. Please link the issue for context.

Checklist

  • [X] You've signed-off your work
  • [X] Your changes are accompanied by tests (if relevant)
  • [X] Your change contains a small diff and is self-contained
  • [ ] You've updated any relevant documentation (if relevant)
  • [X] You've updated the CHANGELOG.md with details about your change under the "Unreleased" section (if relevant, depending on the change, this may not be necessary)
  • [ ] You've versioned your .sql database schema migration according to Flyway's naming convention (if relevant)
  • [ ] You've included a header in any source code files (if relevant)

collado-mike avatar Sep 21 '22 19:09 collado-mike