dbt-sqlserver icon indicating copy to clipboard operation
dbt-sqlserver copied to clipboard

Update the derivation of the test view name

Open Benjamin-Knight opened this issue 1 year ago • 1 comments

Resolves #575 Use the local MD5 macro to create a more unique hash based on the SQL of the test itself

Benjamin-Knight avatar Oct 24 '24 13:10 Benjamin-Knight

Thanks @Benjamin-Knight. Will set some time aside this week to review.

cody-scott avatar Oct 28 '24 15:10 cody-scott

Hi, we're having the same problem. I'm fine with the solution and this can be ignored if it causes too much trouble

Edit: I'm really fine with the solution, just leaving the rest of my world salad here for people with spare time. {{ local_md5(main_sql) }}_{{ range(1300, 19000) | random }} should do the trick and you've got all the variables at hand anyways

Isn't the more common approach to hash the "dbt name" of the test (which should be unique as well)? Or hash plus a random seed in case someone somehow manages to be really crazy with column and table names. You could probably have two tests that run the same sql but yaml logic should forbid that there is the same test twice.

So one would hash relationships_table1_conflict_column1__column2__ref_table2 instead of the underlying SQL.

Cheers Hannes

johannes-becker-bt avatar Oct 31 '24 09:10 johannes-becker-bt

@johannes-becker-bt The pull currently uses a hash and a random seed as you mention for the exact reason you noted.

Using the name of the test does sound like a better hash target, I'll have to test if DBT allows the setting of test names to be the same value, I often override the name of tests to a more user friendly version and DBT would need to ensure this is unique as well for this to work.

Benjamin-Knight avatar Oct 31 '24 09:10 Benjamin-Knight

Only if you feel like it. Main reason I would stick with your solution: we got main_sql available. All other solutions would need much more tinkering in different places (Unless the test name is somewhere in target or something, I'm never sure about that...)

johannes-becker-bt avatar Oct 31 '24 09:10 johannes-becker-bt

Merry Christmas! I tested it with a local macro and it works like a charm. Also it might make sense to combine the solution with the other change regarding tests https://github.com/dbt-msft/dbt-sqlserver/pull/586 Have a great New Year! Hannes

johannes-becker-bt avatar Dec 27 '24 08:12 johannes-becker-bt

^ Yes please! Would love to hit 2 birds with one stone on this

timestescrane avatar Dec 30 '24 18:12 timestescrane

Added an update to resolve #586 as well, I added the update to view name derivation to the unit test materialisations as well.

Benjamin-Knight avatar Jan 02 '25 16:01 Benjamin-Knight

@cody-scott would you be able to give this a look-over this week? Would help unblock some upstream work, well, at work. :)

timestescrane avatar Jan 08 '25 15:01 timestescrane

@Benjamin-Knight would you be able to merge this PR?

timestescrane avatar Jan 14 '25 16:01 timestescrane

I do not have permissions to do that, all checks are now passing that the failing tests were updated via #589 so just waiting on @cody-scott to sign it off now its all passing.

Benjamin-Knight avatar Jan 14 '25 16:01 Benjamin-Knight

@cody-scott would you be able to merge this PR? Getting to a point where having this fix in is going to make my life a lot better at work.

timestescrane avatar Jan 16 '25 16:01 timestescrane