sodadata
`soda scan` should accept a single YAML file or a folder
When running scans using the CLI command soda scan it would be great to run scans from multiple YAML files.
Actually, it only accepts a single file.
This could be useful in all those scenarios where scans are organized into different folders, i.e: one folder for each DWH layer (staging area, integration, datamart, and so on).
Ideally, soda scan should be smart enough to understand if the provided argument represent a YAML file or a folder.
If the provided argument is a file, then run scans in the file.
If the provided argument is a folder, then run all scan defined in YAML files that are in the folder.
That would indeed be useful. Otherwise, we are required to use a command for every scan file that was created by the soda analyze command, which could be tedious when you have a lot of tables.
Something "kind of" smart would be to use this, but this gives you a lot of output:
for scan_file in `ls tables`; do soda scan <my_warehouse>.yml schema_tables/"$scan_file"; done
Would be amazing if soda scan could be pointed to a directory and could run all the included yml files in parallel. The current implementation is limited to running scans in series which is not sustainable when the number of tables grows above a reasonable number.
