Replication, CAP Theorem & Consistency
Master database replication (Primary-Replica, Multi-Primary), synchronous vs asynchronous replication, the CAP Theorem, PACELC model, and consistency models from strong to eventual — the foundations of every distributed database.
Master database replication (Primary-Replica, Multi-Primary), synchronous vs asynchronous replication, the CAP Theorem, PACELC model, and consistency models from strong to eventual — the foundations of every distributed database. This hands-on tutorial focuses on practical implementation of replication, cap theorem & consistency concepts.
Replication, CAP Theorem & Consistency
Replication is the practice of copying data across multiple servers to achieve two goals: high availability (survive server failures) and read scalability (distribute read load). But replication introduces one of distributed systems' hardest problems: consistency.
[!IMPORTANT] The CAP Theorem is one of the most frequently tested topics in system design interviews. You must be able to classify any database as CP or AP and explain the trade-off in context.
Primary-Replica Replication
The most common replication pattern: one Primary (writer) + N Replicas (read-only copies):
Synchronous vs Asynchronous Replication
CAP Theorem
The CAP Theorem states that a distributed system can only guarantee 2 of 3 properties simultaneously:
[!NOTE] In practice, P (Partition Tolerance) is non-negotiable — networks WILL fail. So the real choice is between CP (sacrifice availability to stay consistent) and AP (sacrifice consistency to stay available).
PACELC Model
CAP only covers behavior during partitions. PACELC (Patterson, 2012) extends it to cover normal operations too:
If P (partition) → choose A (availability) or C (consistency)
Else (normal operation) → choose L (latency) or C (consistency)
| Database | During Partition | Normal Operation |
|---|---|---|
| MySQL + Sync Replication | PC | LC (low latency, eventual) |
| Cassandra | PA | EL (low latency, eventual) |
| DynamoDB | PA | EL |
| Spanner (Google) | PC | EC (consistent, higher latency) |
| etcd | PC | EC |
Consistency Models (Strongest → Weakest)
Quiz
Quiz
Question 1 of 2Your payment system processes bank transfers. During a network partition between your two data centers, a transfer is initiated. You decide that DC2 should refuse to process any requests until the partition heals. Which CAP category is this?
Key Takeaways
✅ Primary-Replica is the most common pattern: single write node, multiple read nodes.
✅ Synchronous replication = zero data loss, higher write latency. Async = lower latency, risk of data loss.
✅ CAP Theorem: networks will partition — choose CP (stay consistent, go down) or AP (stay up, tolerate stale data).
✅ Eventual consistency is acceptable for most social features; strong consistency is required for financial data.
✅ PACELC extends CAP — also consider latency vs. consistency trade-offs during normal (non-partition) operation.
Official Resources
- 📖 PostgreSQL Replication — Streaming replication, logical replication
- 📖 Cassandra Consistency Levels — Tunable consistency deep dive
- 📖 AWS DynamoDB Consistency — Eventually vs strongly consistent reads
- 📖 Martin Kleppmann — Designing Data-Intensive Applications — Best book on this subject
What's Next?
You know how to distribute data across servers — but what about distributing large binary files like images and videos?
Next: Blob, Object & File Storage — How AWS S3, HDFS, and distributed file systems store billions of files.