Glossary

A manually triggered backfill initiated from the pipeline overview, as opposed to the automatic backfill that runs during initial onboarding.

A process that copies existing historical data from a source table to the destination. Backfills run alongside ongoing CDC so that both current and historical rows are replicated.

See the Backfill guide for more details.

A deployment model where the data plane runs inside your own cloud account or network while Artie hosts the control plane. This keeps all customer data within your infrastructure.

Change data capture is a continuous process that captures inserts, updates, and deletes from a database’s transaction log. Artie uses CDC to stream changes in near real-time to your destination.

Encrypting column values before writing them to the destination. Values can be decrypted later using either an Artie-managed key or a customer-provided KMS data encryption key (DEK). See Tables.

A filter that replicates all columns except those in a specified list, useful for omitting PII or large unused columns. See Column inclusion and exclusion.

Replacing column values with a SHA-256 hash before writing to the destination. If the pipeline uses a column hashing salt, Artie uses HMAC-SHA256 with that salt instead. The same value and salt always produce the same hash, so hashed columns stay joinable across tables that use the same salt. See Hash certain columns.

A filter that replicates only an explicit list of columns, excluding everything else by default. See Column inclusion and exclusion.

Artie has an in-memory compaction mechanism that reduces the volume of writes sent to the destination by compacting multiple changes to the same row identified by primary key before flushing. By doing so, Artie can reduce the number of writes sent to the destination.

The Artie-hosted layer responsible for the dashboard, pipeline configuration, monitoring, and metrics. No customer row data passes through the control plane. See Architecture.

The environment where all data processing happens: reading from your source, buffering changes in Kafka, and writing to your destination. A data plane is scoped to a cloud provider and region. In BYOC deployments the data plane runs in your own network. See Architecture.

A configurable name override that lets a source table appear under a different name in the destination. See Tables.

Amazon DocumentDB is a fully managed, MongoDB-compatible document database service. Artie replicates data from DocumentDB via change streams and requires an SSH tunnel since DocumentDB only allows access within your VPC. See the DocumentDB source guide.

One of three thresholds (time elapsed, deduplicated row count, or deduplicated byte size) that triggers a flush when any single condition is met. See Flush rules.

The process of writing buffered, in-memory data to the destination as an optimized batch, then committing the Kafka offset and resuming reads. See Flush rules.

A table setting that creates a companion {TABLE}__HISTORY table recording every insert, update, and delete for auditing and point-in-time analysis. See Tables.

An additional destination column (such as a partition or cluster key) included in merge logic so the data warehouse can prune work during merges and improve performance. See Tables.

A pipeline-level setting that compresses the entire Kafka message value in flight. Values are decompressed before parsing, so the destination schema and data are unchanged.

Message compression is mutually exclusive with transparent column compression, so only one can be active per pipeline. See Pipelines.

For high-throughput Snowflake, MotherDuck, and Iceberg tables, loading several flushes into a staging table, then running one merge into the target table, to avoid timeouts on very large flush batches. See Enable multi-step merge.

A configured replication path from a source through Artie to a destination, including which tables to replicate and their individual settings.

An AWS networking feature that provides private connectivity between VPCs without exposing traffic to the public internet. Artie supports PrivateLink as a connection option for sources and destinations hosted in AWS.

A secure, encrypted connection that forwards network traffic between Artie and a database through an intermediary host (such as a bastion or EC2 instance). See SSH tunnel setup.

Automatic management of destination schema changes: adding new columns, optionally dropping columns that were removed upstream after a safety verification period and supporting column type changes.

See the Schema evolution guide for more details.

A table setting that causes Artie to ignore delete events so deleted source rows are not removed from the destination. See Tables.

A destination setting where deleted source rows are retained in the destination and flagged with a delete column (such as __artie_delete) instead of being physically removed.

Splitting rows into time-based physical tables (for example, monthly or daily) while exposing a single unified view for queries. See Soft partitioning.

Optional destination columns prefixed with __artie_ that Artie adds for tracking operations, deletes, source metadata, and static tags. See System columns.

Compressing individual column values in flight so large payloads fit within Kafka’s message size limit. The column value is gzip-compressed and base64-encoded in the Debezium envelope; Artie decompresses it before writing to the destination, so the logical schema is unchanged. This is a per-table, per-column setting available on PostgreSQL sources.

Mutually exclusive with message compression, so only one can be active per pipeline. See Tables.

In soft partitioning, a database view that combines all partition tables using UNION ALL so consumers can query a single logical table name. See Soft partitioning.