jitsu icon indicating copy to clipboard operation
jitsu copied to clipboard

Full refresh for Singer, Airbyte and Go native sources

Open vklimontovich opened this issue 3 years ago • 0 comments

Problem

At the moment, full refresh for subj doesn't work well. It first truncates table, and then starts to fill table once again. It means that during the run data is inconsistent.

Solution

We have a mechanism for transactional full refresh which we use in Jitsu SDK backed sources. We should utilize same thing for all other types of sources.

Cleanup deprecated tables.

vklimontovich avatar Aug 02 '22 15:08 vklimontovich