dataform icon indicating copy to clipboard operation
dataform copied to clipboard

Workspaces and local development

Open ollimandoliini opened this issue 11 months ago • 3 comments

I'm looking for a way to combine development in the GCP Dataform UI using workspaces and more traditional development workflow in a code editor. In both of the use cases, I want to be able to suffix the dataset name with the developer's name. I'm either looking for a way to use workspaces from my local machine with dataform CLI or a way to set the suffix on my local machine without modifying the project wide configuration. Any ideas?

ollimandoliini avatar Feb 27 '25 11:02 ollimandoliini

I'm looking for a nicer version of the same solution.

Here's a bit of a hack that works:

dataform run \
	--actions my_model \
	--schema-suffix $(whoami)

You can substitute whoami for other options.

I just wrap this into a script.

Ideally the CLI would allow you to specify a workspace name so you could inherit all the same overrides without doing any work.

HHammond avatar Mar 11 '25 18:03 HHammond

Bit of a self plug but if you use vscode, you can use Dataform tools extension and pass compiler options similar to how @HHammond has suggested 🙂

Image

ashish10alex avatar Mar 14 '25 14:03 ashish10alex

I'll give it a try!

That extension looks great, the DAG view is much nicer than what I've put together. I appreciate the inline errors and cost estimates too

HHammond avatar Mar 14 '25 18:03 HHammond