OpenMetadata
OpenMetadata copied to clipboard
Add support for the profiler for ingestion time partition in BQ (`_PARTITIONTIME`/`_PARTITIONDATE`)
Context
Profiler and data quality currently supports BQ partitioned table with explicit fields (i.e. fields that are present in the table), though BQ allows users to partition tables by ingestion time using pseudo column _PARTITIONTIME or _PARTITIONDATE.
Solution
Implement support for pseudo column predicate in sampler's and runner's query
A.C.
- Users are able to run profiler and tests against column with ingestion time partition
Working on adding Ingestion Time on the ingestion side.
Added a fix in https://github.com/open-metadata/OpenMetadata/pull/7506