add fivetran_union_relations macro
What change does this PR introduce?
Adds fivetran_union_relations macro. This was necessary for rolling union_data out to Zendesk, which has some source tables that have reserved-keywords for names (ie timezone). In the past, we've solved this error by aliasing the problematic source table, but there's no way to get dbt_utils.union_relations to do so (besides opening a PR on dbt utils or making our own version of the macro).
We already do have our own version of union_relations in this package, but as we learned recently, using the same exact name as a macro in debt_utils can cause failures for customers with certain dispatch configs
This PR also adjusts union_data (in order to work with Netsuite, which is a weird package due to our support of both of its endpoints).
- new optional (none by default)
connector_table_name_overridefield. this is to be used when the definedsourcetable name is different from the actual table name as it appears in the connector/warehouse. This is only a thing for Netsuite2- Added a reference to a new global variable
use_table_name_identifer_override. this was necessary to add for our integration tests (and customers) to still be able to use identifier variables for tables in whichconnector_table_name_overrideis implemented.
- Added a reference to a new global variable
If this PR introduces a new macro, how did you test the new macro?
Ran it with zendesk, hubspot, and netsuite in their feature/add-union-data branches. moreover, i tested this fivetran_utils branch on Shopify_source, which has the union ability already.
If this PR introduces a modification to an existing macro, which packages is the macro currently present in and what steps were taken to test compatibility across packages?
- just added a comment to the
union_relationsmacro to deprecate it in the future
Ran it with zendesk, hubspot, and netsuite in their feature/add-union-data branches. moreover, i tested this fivetran_utils branch on Shopify_source, which has the union ability already.
Did you update the README to reflect the macro addition/modifications?
- [x] Yes
- [ ] No (provide further explanation)