meteor icon indicating copy to clipboard operation
meteor copied to clipboard

Support ability to exclude datasets/tables based on labels in bigquery extractor

Open bsushmith opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. Currently in bigquery extractor, one can exclude datasets/tables based on name. But if the tables/dataset list is huge or if there is a need to exclude based on a pattern like tables starting with xx_ or YYYYMMDD_ values, it gets difficult to maintain or modify meteor extractor recipes every time a new format of exclusion comes into the picture.

Add ability to exclude datasets/tables based on labels so that one can attach labels during the table/dataset creation which will be automatically get excluded in meteor extraction.

Describe alternatives you've considered A regex pattern matching in the name might help for quite a few... but there could be multiple regex patterns that could arise out of this, and it might be cumbersome to maintain / modify in config each time.

bsushmith avatar Jan 19 '23 04:01 bsushmith