The Data Almanac
Learn about change data capture, replication, and modern data systems.
Change Data Capture & Replication
How CDC works in practice, from transaction logs and replication slots to backfills and moving changes into a warehouse.
- ExplainerPostgreSQL 17 Logical Failover Slots: How CDC Survives a Primary PromotionA bullet-point guide to PostgreSQL 17 logical failover slots, the required configuration, and a tested CDC failover drill.
- ExplainerHow Real-Time Data Synchronization Works and Why It Matters in 2026How real-time data synchronization uses CDC to keep warehouses current with production databases in under a minute.
- ExplainerDatabase Backfill Without Downtime: How Online Backfills WorkHow online CDC backfills keep destination tables fully usable during a resync, using dual-write staging and atomic table swaps.
- ExplainerClickHouse Integration: A Practical Guide for Data Teams in 2026Learn how to integrate ClickHouse Cloud with Postgres, MySQL, and more using CDC. Covers table engines, batching, and real-time pipeline setup.
- How-toAI Data Integration: How to Build Real-Time Pipelines for AI SystemsReal-time AI data integration: how CDC-based pipelines eliminate stale data in AI agents, RAG systems, and fraud detection models.
- ExplainerStream DynamoDB to Redshift with CDCLearn how to stream DynamoDB to Redshift with CDC for real-time analytics, faster reporting, and scalable pipelines for all teams.
- How-toHow to Replicate MySQL to Redshift in Real TimeLearn how MySQL CDC streams changes to Redshift in real time, with setup steps, MERGE patterns, and production tips.
- ExplainerReal Time Data Streaming Architecture: Patterns, Components & Design PrinciplesLearn how real-time data streaming architecture works, including CDC, pipeline components, design principles, and best practices.
- ExplainerPostgreSQL to Snowflake ReplicationStream PostgreSQL to Snowflake in real time with CDC, schema evolution, and low-latency replication using Artie.
- How-toMySQL Replication Best Practices: Security, Performance, and Safe FailoverLearn MySQL replication best practices for security, performance, schema changes, lag reduction, and safe GTID-based failover.
- ExplainerPostgreSQL Replication Slots: Create, Monitor, and DropLearn how to create, monitor, and drop PostgreSQL replication slots. Includes SQL examples for checking WAL size, replica identity, and slot troubleshooting.
- ExplainerUnbounded WAL Growth in Idle AWS RDS Logical ReplicationUnbounded WAL growth on idle AWS RDS: why logical replication slots fill storage and how to prevent it.
- How-toHow to Build Audit Logs Using CDC and SCD Type 2Build audit logs with CDC and SCD Type 2. Learn a step-by-step approach to track every change with full history.
- How-toPostgres Replication Slot 101: How to Capture CDC Without Breaking ProductionLearn how to use Postgres replication slots for CDC safely without risking storage issues or production failures.
- How-toWhy TOAST Columns Break Postgres CDC and How to Fix ItTOAST columns can break Postgres CDC. Learn why jsonb turns NULL and how to fix it without bloating WAL.
- ExplainerPostgres CTID Scanning for Faster BackfillsPostgres CTID scanning enables 10x faster backfills for large tables. Learn how CTID works and boosts performance.
- ExplainerPostgres Write-Ahead LogsPostgres write-ahead logs explained: how WAL works, key settings, and best practices for logical replication.
- How-toMySQL Cascading Changes and Best PracticesLearn MySQL cascading changes and best practices to maintain referential integrity across related tables.
- ExplainerPostgres Table Replica IdentityUnderstand Postgres table replica identity, how it impacts logical replication, and why it matters for CDC.
- ExplainerHistory TableLearn how History Tables capture every CDC change with timestamps and operation types for full audit and replay.
- ExplainerIntroduction to Database ReplicationIntroduction to database replication: ensure data accuracy, low latency, redundancy, and reliable access.
- ExplainerPrevent WAL Growth on AWS Postgres RDSPrevent WAL growth on AWS Postgres RDS. Learn how to avoid replication slot overflow and storage issues.
- ExplainerWhy ELTs Are Here to StayWhy ELTs are here to stay: how modern data stacks shift transformations into the warehouse for flexibility.