Feat!: forward-only changes schema modification check

[treysp]

Tasks

• [x] Model defaults code, tests
• [x] Plan selector code, tests
• [x] Plan time code, tests
• [x] Run time code, tests
• [x] Docs
• [ ] dbt

Feature description

Some forward-only changes require schema modifications that drop existing data (e.g., changing a column type from str to int). If a user inadvertently makes such a change, they will lose existing data. This feature identifies destructive changes and optionally warns/errors before they occur to avoid accidental data loss.

The feature introduces a new model kind property on_destructive_change with valid values ignore, warn, and error (default is error). The on_destructive_change property can be specified as a model default.

Users may want to temporarily allow a destructive change to a forward-only model. The feature introduces to the plan command a new --allow-destructive-model selector to temporarily allow destructive changes.

Of note

• Defaults

  • on_destructive_change is a property of a model kind
    • Only incremental models are checked for destructive changes, and their default is ERROR
    • Non-incremental models default is IGNORE
  • Users may specify a default in the model_defaults configuration key - it will only apply to incremental models, as non-incremental models are always ignored

• Plan time check

  • Uses SchemaDiffer and engine-specific rules to determine whether a change is destructive.
  • We pass the current engine's SchemaDiffer into the plan builder to perform the comparison.
  • Evaluating a change requires that columns_to_types is known.
  • Evaluates both direct and indirect changes to a model. A complete evaluation requires knowing columns_to_types for the model and all directly modified ancestors.

• Run time check

  • Uses the set of Alters passed from SchemaDiffer to alter_table() to determine whether a change is destructive.