data-diff icon indicating copy to clipboard operation
data-diff copied to clipboard

state:modified --dbt

Open dlawin opened this issue 2 years ago • 3 comments

Is your feature request related to a problem? Please describe. I'd like to use the --select state:modified selector with datadiff

Describe the solution you'd like Select the same models as dbt and diff them

dlawin avatar Jul 06 '23 17:07 dlawin

In order to accomplish this we'll need to require the --state flag like dbt, and pass the location of that manifest along to the dbt package here:

  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/__main__.py", line 311, in main
    dbt_diff(
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/dbt.py", line 79, in dbt_diff
    models = dbt_parser.get_models(dbt_selection)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/data_diff/dbt_parser.py", line 157, in get_models
    return self.get_dbt_selection_models(dbt_selection)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

e.g. current exception:

  File "/Users/dan/repos/demo/env/lib/python3.11/site-packages/dbt/graph/selector_methods.py", line 556, in search
    raise DbtRuntimeError("Got a state selector method, but no comparison manifest")
dbt.exceptions.DbtRuntimeError: Runtime Error
  Got a state selector method, but no comparison manifest

dlawin avatar Jul 06 '23 18:07 dlawin

As a workaround before this is added, the DBT_STATE env var can be set to the folder with the "prod" manifest

✗ export DBT_STATE=/Users/dan/repos/my_folder_with_prod_manifest

dlawin avatar Nov 10 '23 20:11 dlawin

As a workaround before this is added, the DBT_STATE env var can be set to the folder with the "prod" manifest

✗ export DBT_STATE=/Users/dan/repos/my_folder_with_prod_manifest

note that this expects the prod manifest to be named manifest.json

dlawin avatar Nov 14 '23 18:11 dlawin