versatile-data-kit
versatile-data-kit copied to clipboard
vdk-oracle: test fetch-ingest scenario with different source and destination
Overview
We should be able to support the following scenario, because the default db type and ingestion method are configured separately. However, this needs to be verified and functional tests added.
Scenario:
Create a simple job with the following steps
Step 1: read data from Oracle Step 2: clean up data (optional for this test) Step 3: send cleaned data to another database (impala or postgres)
config.ini should have something like
[vdk]
DB_DEFAULT_TYPE=oracle
INGESTION_DEFAULT_TYPE=impala
Acceptance criteria
- Above scenario is tested
- A functional test is added for it
- An example is added in the documentation
- Separate tickets are opened for any gaps that are discovered