MySQL parallel backfill (integer primary keys)

Backfills are often the longest part of onboarding a new pipeline and a major source of time-to-value delays for large tables.

Artie now supports parallel backfills for MySQL tables with integer primary keys. During the initial load, Artie automatically chunks a table into primary-key ranges and backfills those ranges concurrently instead of performing a single-threaded scan.

For example, if orders.id ranges from 1 to 200,000,000, Artie can split that range into multiple chunks and backfill them in parallel.

Why this matters

• Dramatically reduce wall-clock time for large initial loads
• Turn one slow sequential scan into multiple concurrent range reads and writes
• Keep backfill logic simple and deterministic using primary-key ranges
• Enable much faster onboarding for large MySQL tables, especially when replicating from read replicas, without changing schema or writing custom partitioning logic