dbt-data-reliability icon indicating copy to clipboard operation
dbt-data-reliability copied to clipboard

elementary column anomalies tests fail on redshift with error 'Relation name is longer than 127 characters'

Open MiConnell opened this issue 2 years ago • 1 comments

Describe the bug elementary column_anomalies tests are unable to run on Redshift due to table naming length restrictions

To Reproduce Steps to reproduce the behavior:

  1. add elementary.column_anomalies tests to columns in a dbt model that builds in Redshift
columns:
      - name: id
        tests:
          - elementary.column_anomalies:
              column_anomalies:
                - null_count
                - missing_count
                - average
        description: ''
        meta:
          hidden: true
  1. test the model with dbt test -s my_model

error occurs:

Runtime Error in test elementary_column_anomalies_fact_table_null_count__missing_count__average__success_count (models/marts/fact_table.yml)
Relation name '"test_7ffaec620b_elementary_column_anomalies_fact_table_null_count__missing_count__av__metrics__tmp_20240423000522692548"' is longer than 127 characters

Expected behavior Tests run successfully and elementary is able to create temp tables

Environment (please complete the following information):

  • dbt package Version: 0.14.1

Additional context dbt running in prod redshift

MiConnell avatar Apr 23 '24 18:04 MiConnell

It looks like the problem is it's adding both single and double quotes. the relation name '"<table_name>"' is longer than 127 characters is accurate as it's actually 129 characters. changing this line to 125 fixes it.

MiConnell avatar Apr 23 '24 21:04 MiConnell

Hi @MiConnell ! Thanks for opening this issue and the associated PR, and sorry for the delayed response. I believe the double quoting is likely a bug here, but nonetheless I don't see an issue with reducing the number of characters to 125 for now so I'm going to approve and merge it.

haritamar avatar May 15 '24 14:05 haritamar