`pre_operations` compiles to `;` even if empty
Hello,
I use Dataform to create a table in BigQuery. In the SQLX file I have some pre_operations. Everything in pre_operations is dependent on the value of a custom compilation variable. Based on the value of that variable the pre_operations could end up being deactivated and thus compiling to an empty section in the SQL that is sent to BigQuery.
Unfortunately, in this case, the Dataform compiler still inserts a ; after an empty string. This causes the following error in BQ: Error: Syntax error: Expected end of input but got ";" at [2:1].
At the moment I’m circumventing this by placing in the else section of the when() function that checks the value of the custom compilation variable a filler sql statement like SELECT [1, 2, 3] as numbers.
This is sub-optimal. The ideal behaviour for the Dataform compiler would be to avoid placing a ; if pre_operations is empty.
We've now fixed GCP Dataform to no longer generate these extra semicolons, which should solve this issue for most people.
However, it would also be nice (for general cleanliness) to change the compiler to remove pre/post operations which are just blank space.
This seems to have been done now, empty pre and post ops are ignored:
config {
type: "table",
}
pre_operations {
}
SELECT 1
post_operations {
}
becomes the compiled action of:
{
"type": "table",
"target": {
"schema": "dataform_core_testing",
"name": "tmp",
"database": "cloud-dataform-testing"
},
"query": "\n\nSELECT 1\n\n\n",
"disabled": false,
"fileName": "definitions/tmp.sqlx",
"canonicalTarget": {
"schema": "dataform_core_testing",
"name": "tmp",
"database": "cloud-dataform-testing"
},
"enumType": "TABLE"
}
(the pre_ops and post_ops fields are unpopulated) https://github.com/dataform-co/dataform/blob/b5368e04a047c6f2c958b6dd14d9bd5f872e4e0f/protos/core.proto#L142