dbt-codegen
dbt-codegen copied to clipboard
`generate_base_model()` doesn't work across databases on Redshift
Describe the bug
If your redshift connection is to the database dev but your source is on a separate database sources - then the generate_base_model() macro does not work due to https://github.com/dbt-labs/dbt-redshift/issues/736
Steps to reproduce
- Connect your redshift project to a database
dev. - Specify a source that is on a separate database:
# models/sources.yml
version: 2
sources:
- name: my_sources
database: sources_only
schema: public
tables:
- name: source_foo
- Use the
codegen.generate_base_model()macro on that source and see that we are not able to list the columns since https://github.com/dbt-labs/dbt-redshift/issues/736 meant thatadapter.get_columns_in_relation()does not work across databases for Redshift. (Here is where this package usesget_columns_in_relation())
$ dbt debug
...
02:14:25 Connection:
...
02:14:25 user: root
02:14:25 port: 5439
02:14:25 database: dev
...
$ dbt run-operation generate_base_model --args '{"source_name": "my_sources", "table_name": "source_foo"}'
02:15:00 Running with dbt=1.7.9
02:15:01 Registered adapter: redshift=1.7.1
02:15:03 Found 1 model, 1 source, 0 exposures, 0 metrics, 594 macros, 0 groups, 0 semantic models
with source as (
select * from {{ source('my_sources', 'source_foo') }}
),
renamed as (
select
from source
)
select * from renamed
Expected results
We should be able to get column information across databases and write them into the renamed cte.
Actual results
See above.
Screenshots and log output
See above.
System information
The contents of your packages.yml file:
Which database are you using dbt with?
- [ ] postgres
- [x] redshift
- [ ] bigquery
- [ ] snowflake
- [ ] other (specify: ____________)
The output of dbt --version:
Core:
- installed: 1.7.9
- latest: 1.7.10 - Update available!
Your version of dbt-core is out of date!
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
- databricks: 1.7.3 - Update available!
- bigquery: 1.7.3 - Update available!
- snowflake: 1.7.2 - Up to date!
- redshift: 1.7.1 - Update available!
- postgres: 1.7.9 - Update available!
- spark: 1.7.1 - Up to date!
At least one plugin is out of date or incompatible with dbt-core.
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
The operating system you're using:
macOS
**The output of python --version:
Python 3.11.2
Additional context
This won't be fixed in this package until some point we actually fix it in the redshift adapter itself in https://github.com/dbt-labs/dbt-redshift/issues/736