Backfills
Going over how backfills work in Artie
Overview
The following circumstances may trigger a :
Deploying for the first time
When you launch a new Artie pipeline, we will backfill all the tables.
Adding new tables
We will backfill the new tables you have added.
Adhoc backfills
Trigger a backfill for a specific table on demand from the pipeline overview page.
How it works
Read replica support
You can specify a read replica for backfills to minimize any impact on your primary database.
Parallel processing
Artie backfills by having parallel processes per table (default is 5). This speeds up the copying process without adding too much load to your database.
Real-time sync
While backfilling, Artie will continue to read your database logs, capturing all new changes in real-time into our Kafka topics. Once the backfill is complete, Artie process will then switch to consume from the Kafka topics.