What is WAL?
What is WAL?
Write-Ahead Logging (WAL) is Postgres’s built-in mechanism for ensuring data integrity and enabling change data capture (CDC). Here’s how it works:
- Every database change is first written to the WAL
- The changes are then applied to the actual database files
- This approach ensures data durability and enables reliable replication
- Replication slot overflow
- Database storage exhaustion
- Potential database downtime
Why is WAL growth an issue only on RDS?
Why is WAL growth an issue only on RDS?
TL;DR: 🚨 AWS RDS uses internal “heartbeats” that generate WAL entries every 5 minutes, which can cause significant storage issues on idle databases.Detailed Explanation:As explained by Gunnar Morling in his blog post, AWS RDS writes a heartbeat to an internal
rdsadmin
table every 5 minutes. Here’s why this matters:- Each WAL segment is 64MB by default
- Each heartbeat creates a new WAL segment
- On idle databases, this means:
- 64MB of WAL growth every 5 minutes
- ~18.4GB of WAL growth per day
- Potential replication slot overflow if left unchecked
- Test databases
- Low-traffic environments
- Idle databases
🛡️ Preventing WAL Growth with Heartbeats
This solution is specifically designed for low-traffic or idle databases. Active databases don’t need this feature as their WAL naturally resets with regular data changes.Setup Steps
- Create and configure the heartbeat table:
- Enable heartbeats in your pipeline’s advanced settings.
Troubleshooting Guide
If you’re still experiencing WAL growth after enabling heartbeats, check these common issues:1
Table Existence
Verify
test_heartbeat_table
exists in your database2
Publication Configuration
3
Permission Issues
Confirm the service account has proper write permissions
4
Long-Running Queries
Additional Best Practices
Additional Best Practices
To further protect your RDS instance, implement these measures:
-
Monitoring
- Set up alerts for
free_storage_space
- Monitor WAL growth rates
- Set up alerts for
-
Database Configuration
- Set appropriate
statement_timeout
- Configure
max_slot_wal_keep_size
(default is -1 for unlimited) - Enable storage autoscaling (AWS Guide)
- Set appropriate
Useful Diagnostic Commands
Useful Diagnostic Commands