Bashir Sadjad
Bashir Sadjad
The "Recreate Views" functionality of the controller, does not recreate the flat Parquet files for views. Steps to reproduce: - Do a "Full Run". - In the created DWH root,...
We have experimented with some static code analyzers like SonarQube (e.g., in #704, #706, #707) but these are not adequate, especially for catching `NullPointerException`s. Also we have some `@Nullable` annotations...
This is a follow-up of #1003 and in particular [this comment](https://github.com/google/fhir-data-pipes/issues/1003#issuecomment-2384862734). Instead of trying to record the resource-type in the `id` field, we should add a `resourceType` field to the...
## Description of what I changed This addresses yet another source of flakiness in our E2E runs: Looking at some of the recent failures, it seems the initial count of...
The context for this issue is [this comment](https://github.com/google/fhir-data-pipes/issues/1431#issuecomment-3333344033). The user was not getting all the resources in the DWH and the root cause seems to be failing FHIR Search queries....
There seems to be a mismatch between how we create the Avro schema for a collection in a ViewDefinition [here](https://github.com/google/fhir-data-pipes/blob/b1d77989993cabe865eeb3545279b9e585040b81/pipelines/common/src/main/java/com/google/fhir/analytics/view/ViewSchema.java#L252) vs how we set collection values in an Avro record...
We should do pylint on all of our Python modules, namely, synthea-hiv (generator and uploader), e2e-tests, querygen, etc. in the E2E build and fail if there are major issues. We...
## Description of what I changed Please note that this is on top of PR #1402. ## E2E test TESTED: Relying on e2e/unit tests. ## Checklist: I completed these to...
When generating Parquet-based materialized flat views for a large dataset (MIMIC), the pipeline fails with out-of-memory issues even with a huge amount of available memory (100+ GB). This happens while...