stevenayers-bge
stevenayers-bge
### Describe the feature Add support for DLT Expectations: [Manage data quality with Delta Live Tables ](https://docs.databricks.com/en/delta-live-tables/expectations.html) These could utilize the [constraints](https://docs.getdbt.com/reference/resource-properties/constraints) resource property for syntax: ```yaml models: - name:...
Hi all, I want to use `load_as_spark`, but instead of saving the config credentials as a file, I want to pull them from a secrets manager (Azure Vault, AWS SecretsManager,...
When writing Databricks Asset Bundles, it would be great if we could define things like job cluster configuration as one config block: ```yaml targets: dev: variables: spark_version: 14.3.x-scala2.12 pool_id: 12334...
## Description Databricks Asset Bundle configuration does not recognize `cluster_mount_info` as a valid field name, so you cannot provision job clusters with it. ```yaml jobs: my_job: job_clusters: - job_cluster_key: cluster...
NFS mounts are an experimental feature: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster#cluster_mount_info-blocks-experimental They mount at the path `/db-mnt/` on the cluster but you cannot access this path using `dbutils`.
More detail here: - https://github.com/databricks/databricks-sdk-go/issues/866 - https://github.com/databricks/cli/issues/1306 The API spec for clusters contains a field `cluster_mounts_infos`, an experimental feature already supported by the terraform provider: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster#cluster_mount_info-blocks-experimental But it's missing from...
More details can be found here: https://github.com/databricks/cli/issues/1306 The OpenAPI spec used to generate the Go SDK is missing `cluster_mount_infos` so it is not added to `compute.ClusterSpec` https://github.com/databricks/databricks-sdk-go/blob/a823ca32fc4199d8cf2269b78cfe89331b4b688a/service/compute/model.go#L1090 The field is...
**Is your feature request related to a problem? Please describe.** I recently wanted to update the `elementary.clean_elementary_temp_tables()` in the on-run-end hook so that it would simply `DROP SCHEMA CASCADE`; and...
### Describe the feature Add support for the new `MERGE WITH SCHEMA EVOLUTION` syntax available on the current and preview channel SQL Warehouse versions. https://docs.databricks.com/en/delta/update-schema.html#schema-evolution-syntax-for-sql. This is especially useful if...
### Describe the feature Support Column Masking as part of the YAML definition: ```sql {% macro create_hash_function() %} CREATE OR REPLACE FUNCTION my_catalog.my_schema.hash_mask(raw_value STRING) RETURN sha2(raw_value, 256); {% endmacro %}...