TobikoData
Environments for audits
The problem I faced was 2 fold.
I wanted to update an audit and test it. The audit was user defined and involved joins to other models within the project, some of which had also been modified.
- The model reference was only replaced for the physical table that referenced the audit when using my environment. Other models were not substituted and therefor used the promoted views and the audit failed to run because of it.
- I couldn't run audits from the CLI for a particular environment as it isn't an argument or option.
The model reference was only replaced for the physical table that referenced the audit when using my environment. Other models were not substituted and therefor used the promoted views and the audit failed to run because of it.
One way to achieve this is to explicitly enumerate dependencies of your model in the depends_on attribute of the model definition. This way all references will be resolved correctly
Ahh what it looks like I can do is set it to be a standalone audit and then I can set depends_on within the audit.
It looks like this functionality is currently undocumented.
This will be an acceptable workaround for my audit as it will be non-blocking in any case. But I assume that if I ever wanted this to be blocking then it would be problematic again.
it would be problematic again.
Why though? Didn't setting the depends_on on the model for which the audit runs help?
The audit is applied to a more raw (bronze) source of data (not quite an external model) but within the audit I put some joins to refined (silver/gold) data sources that are populated later in the DAG. Those joins are not necessary for modelling the raw data but are relevant for excluding some records from the audit.
My current workaround is to create a view to apply the logic that the audit is then reference in. But the view is then another asset that needs maintaining. I can give more context in Slack but I think this is as distilled as I can get here.