Skip Backfill for New Tables
Select Skip backfill when adding a new table to an existing pipeline to start streaming new CDC changes without a historical load.
When adding a new table to an existing pipeline, a full backfill isn’t always needed. Many teams already have historical data loaded, or they’re incrementally onboarding tables as part of a phased migration. Until now, Artie always kicked off a backfill by default - which worked well for most workflows, but added unnecessary time and compute for cases where history was already in place. This update gives you finer control with a lighter touch.
You can now choose to skip the historical backfill when adding a new table. If your warehouse already contains the table’s past data - like a transactions table you loaded earlier - simply select Skip backfill during deployment. Artie will begin streaming new CDC changes immediately, without performing any historical load.
Why this matters:
- Avoids unnecessary backfills when history already exists
- Cuts down on compute usage and speeds up table onboarding
- Helps teams moving through partial or staged migrations
- Reduces time to first CDC events for newly added tables
- Gives more control over how tables join existing pipelines