versatile-data-kit icon indicating copy to clipboard operation
versatile-data-kit copied to clipboard

vdk-oracle: test fetch-ingest scenario with different source and destination

Open DeltaMichael opened this issue 1 year ago • 0 comments

Overview

We should be able to support the following scenario, because the default db type and ingestion method are configured separately. However, this needs to be verified and functional tests added.

Scenario:

Create a simple job with the following steps

Step 1: read data from Oracle Step 2: clean up data (optional for this test) Step 3: send cleaned data to another database (impala or postgres)

config.ini should have something like

[vdk]
DB_DEFAULT_TYPE=oracle
INGESTION_DEFAULT_TYPE=impala

Acceptance criteria

  1. Above scenario is tested
  2. A functional test is added for it
  3. An example is added in the documentation
  4. Separate tickets are opened for any gaps that are discovered

DeltaMichael avatar Feb 19 '24 08:02 DeltaMichael