dbt_artifacts icon indicating copy to clipboard operation
dbt_artifacts copied to clipboard

[Feature]: data cleanup hook

Open andy-clapson opened this issue 11 months ago • 1 comments

Overview

The artifacts tables can get quite large with time, so some aspect of "rotation" or cleanup would be great. Something like a config-based hook the cleanup all data older than days (or perhaps a certain row number or table size) would be great.

What problem would this solve?

Manage data volume

Would you be willing to contribute?

  • [x] Yes - I'm on it!
  • [ ] Yes, but I'd need help getting started.
  • [ ] No

andy-clapson avatar May 13 '25 02:05 andy-clapson

Yes! I just ran into the issue that it is impossible to do a dbt run -s dbt_artifacts.* --full-refresh because the full_refresh property is forced to false in the dbt_project.yml of dbt_artifacts, making it impossible to clean up those tables...

I will make a separate question about this.

Siete-F avatar May 14 '25 11:05 Siete-F