dataform icon indicating copy to clipboard operation
dataform copied to clipboard

uniqueKey assertions generate invalid SQL for incremental tables in v3.0.2

Open stevenleggdfe opened this issue 1 year ago • 2 comments

To reproduce, put the .sqlx file below in a Dataform repo with dataform-core v3.0.2 installed:

config {
  type: "incremental",
  assertions: {
    uniqueKey: ["foo"]
  }
}

SELECT 1 as foo, 2 AS bar

This generates the SQL below for the assertion (view):

SELECT
  *
FROM (
  SELECT
    TableAssertionsConfig,
    COUNT(1) AS index_row_count
  FROM `project-name.dataform.uniquekey_bug_test`
  GROUP BY TableAssertionsConfig
  ) AS data
WHERE index_row_count > 1

This in turn generates the compilation error "Unrecognized name: TableAssertionsConfig at [6:5]."

This bug does not occur when:

  • An earlier version of dataform-core is used - function is as expected in v3.0.0 and not visible because of a different bug (now fixed) in v3.0.1.
  • The uniqueKey assertion is applied to a non-incremental table
  • uniqueKeys: [["foo"]] is used instead of uniqueKey: ["foo"] - this is a possible workaround if others are experiencing this, but not a fix I feel as uniqueKey is supported and documented here: https://cloud.google.com/dataform/docs/assertions#built-in .

stevenleggdfe avatar Aug 12 '24 11:08 stevenleggdfe

The same issue seems to be present for models of type view

mattimoller avatar Aug 20 '24 13:08 mattimoller

I'm seeing this on models of type view as well.

Updated: This error occurs in v3.0.2 and v3.0.1. v3.0.0 does not throw this error.

blundin avatar Aug 23 '24 20:08 blundin

The issue was fixed by #1836

Ceridan avatar Nov 25 '24 14:11 Ceridan