dataform icon indicating copy to clipboard operation
dataform copied to clipboard

Feature Request - Dependency Chain Export

Open Murry-Ovo opened this issue 4 years ago • 3 comments

It would be great to be able to enter a series of tags, actions or select 'whole project' and be able to export the dependency chain that is created for these runs.

Murry-Ovo avatar Jun 21 '21 13:06 Murry-Ovo

Can you clarify how you'd like this to be exported? Is dataform compile --json sufficient?

BenBirt avatar Jun 28 '21 09:06 BenBirt

I was thinking of a more visual representation, similar to the graphical view in the run logs, however a JSON output would also be useful

Murry-Ovo avatar Jun 28 '21 10:06 Murry-Ovo

This would be a fun feature to make. CLI based text representations would be a bit ugly, because of the limits of scrolling and scaling. I think it would be better to use graphviz to create an image instead. For example:

  • Add a new CLI option for --dot
  • If that option is set, convert the compiled graph JSON format https://github.com/dataform-co/dataform/blob/c0d1a7400f4aed74f90e032149561a3027df0ea4/cli/index.ts#L329 to a dot file instead
    • A dotfile would instead like digraph { dataset.table1 -> dataset.table2 }
  • If the --dot option is set, print the dot file output rather than the json output.
  • Users can then use that dotfile as they wish - probably with the graphviz CLI https://graphviz.org/doc/info/command.html
    • Alternatively we could bundle graphViz in with the CLI, and make the image on demand https://github.com/ts-graphviz/node

Ekrekr avatar Mar 26 '24 15:03 Ekrekr