dataform icon indicating copy to clipboard operation
dataform copied to clipboard

feat: defer to prod

Open AntoineGiraudKeyrus opened this issue 1 year ago • 2 comments

Hi,

Could dataform add a defer to prod option ? i'd help to init our dev env' without having to clone/recreate all our parents tables when editing 1 child table !

resolving in compute, storage & time savings while developping

I'd help us a lot @Homeserve 👷‍♂️

DBT equivalent 🙏 image

AntoineGiraudKeyrus avatar Sep 11 '24 13:09 AntoineGiraudKeyrus

from GCP's dataform issue #366123439

How this might work:

  • dataform's GCP UI : toggle option to choose for "prod" database for deferal in dev environements
  • dataforms's CLI : dataform run --actions ventes+ achats+ --defer prod_profile

If applicable, reasons why alternative solutions are not sufficient:

Without defer to prod here are the options :

  • rebuild ALL parent tables -> storage & compute costs $$
  • clone all parents schemas & tables as views -> ~5min init for 360 tables in 3 schemas create view xxx-dev.xxx.xxx as select * from xxx.xxx.xxx

AntoineGiraud avatar Sep 12 '24 12:09 AntoineGiraud

Bumping this issue.

As a newcomer to dataform from dbt this is something I would use every day.

Defer saves a lot of money in compute and makes development processes much much faster. It's a strong value add.

HHammond avatar Mar 11 '25 18:03 HHammond