Databricks bundle validation doesn't catch cluster settings when using serverless compute is enabled in pipeline resource

[danielsteman]

Describe the issue

databricks bundle validate passes when both serverless is enabled and cluster settings are set for the same pipeline.

Steps to reproduce the behavior

Please list the steps required to reproduce the issue, for example: Create a bundle with the default python template and overwrite databricks.yml with this:

bundle:
  name: my_project
resources:
  pipelines:
    some-dlt:
      name: some-dlt
      libraries:
        - notebook:
            path: src/dlt_pipeline.ipynb
      channel: preview
      clusters:
        - label: default
          num_workers: 1
      serverless: true
targets:
  dev:
    mode: development
    default: true
    workspace:
      host:
  prod:
    mode: production
    workspace:
      host:
      root_path:
    run_as:
      user_name:

Expected Behavior

Return an error that indicates that it's not possible to set cluster settings and also enable serverless

Actual Behavior

No error is returned. When the bundle is deployed, an error is returned.

OS and CLI version

❯ d -v Databricks CLI v0.225.0

MacOS: 14.6.1 (23G93)

Is this a regression?

Not sure

Debug Logs

❯ d bundle validate Warning: expected a string value, found null at targets.dev.workspace.host in databricks.yml:20:12

Warning: expected a string value, found null at targets.prod.workspace.host in databricks.yml:24:12

Warning: expected a string value, found null at targets.prod.workspace.root_path in databricks.yml:25:17

Warning: expected a string value, found null at targets.prod.run_as.user_name in databricks.yml:27:17

❯ d bundle deploy Uploading bundle files to /Users/.../.bundle/my_project/dev/files... Deploying resources... Updating deployment state... Deployment complete! Warning: expected a string value, found null at targets.dev.workspace.host in databricks.yml:20:12

Warning: expected a string value, found null at targets.prod.workspace.host in databricks.yml:24:12

Warning: expected a string value, found null at targets.prod.workspace.root_path in databricks.yml:25:17

Warning: expected a string value, found null at targets.prod.run_as.user_name in databricks.yml:27:17

Error: terraform apply: exit status 1

Error: cannot create pipeline: You cannot provide cluster settings when using serverless compute.

with databricks_pipeline.some-dlt, on bundle.tf.json line 37, in resource.databricks_pipeline.some-dlt: 37: }

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.