Database replication means maintaining copies of the same database data on multiple database nodes.
Instead of having:
Application
|
v
[ One Database ]we can have:
+----------------+
| Application |
+-------+--------+
|
v
+-------------+
| Primary |
+------+------+
|
Replication stream
|
+-----------+-----------+
| |
v v
+-----------+ +-----------+
| Replica | | Replica |
+-----------+ +-----------+Replication is primarily used for:
Replication is different from sharding.
Replication copies data.
Sharding splits data.
That distinction is extremely important in system design.
Imagine an application with one database:
Users
|
v
Load Balancer
|
v
Application Servers
|
v
+----------------+
| PostgreSQL |
| Primary |
+----------------+This creates several problems.
If the database crashes:
Application
|
X
Database DOWNThe application may become unavailable.
Replication gives us another copy.
+-------------+
| Primary |
+------+------+
|
v
+-------------+
| Replica |
+-------------+If the primary fails, the replica can potentially be promoted.
Suppose the application receives:
If every operation goes to one database:
10,000 reads
+
1,000 writes
|
v
One DatabaseThe database can become a bottleneck.
With read replicas:
+-------------+
Writes -----------> | Primary |
+------+------+
|
Replication
|
+-------------+-------------+
| |
v v
+-----------+ +-----------+
Reads->| Replica 1 | Reads-> | Replica 2 |
+-----------+ +-----------+Reads can be distributed across replicas.
A common replication architecture has:
The primary accepts writes.
Replicas maintain copies of the primary's data.
Example:
Writes
|
v
+-------------+
| Primary |
+------+------+
|
Replication
|
+-----------+-----------+
| |
v v
+-------------+ +-------------+
| Replica 1 | | Replica 2 |
+-------------+ +-------------+
^ ^
| |
+------- Reads ----------+Different systems use different terminology:
| Term | Meaning |
|---|---|
| Primary | Node responsible for writes |
| Replica | Copy receiving replicated data |
| Leader | Another common name for primary |
| Follower | Another common name for replica |
| Read replica | Replica primarily used for reads |
| Standby | Replica maintained for availability/failover |
The exact terminology varies by database technology.
You may see:
Leader -> Followersinstead of:
Primary -> ReplicasConceptually they are often similar.
For example:
Leader
|
+----> Follower 1
|
+----> Follower 2The important thing is not the terminology.
The important questions are:
These are system-design questions.
A simplified write flow:
Client
|
| WRITE
v
Primary
|
| 1. Persist change
|
| 2. Generate replication record
v
Replication Stream
|
+---------> Replica 1
|
+---------> Replica 2The exact mechanism depends on the database.
A database may replicate:
A useful conceptual model is:
Application
|
v
Primary
|
v
Change Log
|
+----------+
| |
v v
Replica A Replica BThe change log allows replicas to replay changes.
In synchronous replication, the primary waits for one or more replicas to acknowledge replication before considering the write sufficiently committed according to the configured policy.
Conceptually:
Client
|
v
Primary
|
+------> Replica
| |
| ACK
| |
+<---------+
|
v
Client ACKThe replica is much less likely to be behind when the primary acknowledges the write.
The write now depends on another node and the network between them.
If the replica is slow:
Primary
|
| waiting
v
Replica
|
| slow
v
Higher write latencySo synchronous replication can improve durability/consistency characteristics at the cost of latency and availability under certain failure conditions.
With asynchronous replication, the primary can acknowledge the client without waiting for the replica to confirm that it has applied the change.
Client
|
v
Primary
|
+-----------------> Replica
|
v
ACK ClientThe replica may receive the change shortly afterward.
Lower write latency and less dependency on replica/network availability.
Replication lag can occur.
If the primary fails before a change reaches a replica, that replica may not contain the newest data.
| Property | Synchronous | Asynchronous |
|---|---|---|
| Primary waits for replica acknowledgment | Usually yes | No |
| Write latency | Higher | Lower |
| Replica freshness | Usually stronger | Can lag |
| Failure sensitivity | Higher | Lower |
| Cross-region use | Expensive in latency | Common |
| Potential data loss after primary failure | Lower depending on configuration | Possible |
Neither is universally better.
The choice depends on requirements.
Some systems provide configurations between fully synchronous and fully asynchronous behavior.
A simplified model:
Primary
|
+---- Replica A
|
+---- Replica BThe primary may wait for acknowledgment from at least one replica while other replicas continue asynchronously.
This can provide a compromise between:
The exact behavior is database-specific.
Replication lag is the delay between a change being committed or generated on the primary and becoming available on a replica.
Example:
Primary:
User balance = ₹500
Replica:
User balance = ₹400The primary was updated first.
The replica is temporarily stale.
Possible causes include:
Primary ----slow network----> ReplicaThe replica receives changes faster than it can apply them.
The replica cannot persist changes quickly enough.
A large transaction can take significant time to replicate/apply.
If the replica is overloaded with reads, replication processing may compete for resources.
Geographic distance introduces network latency.
Read replicas are replicas used to serve read traffic.
Example:
+-------------+
| Primary |
+------+------+
|
Replication
|
+-------------+-------------+
| |
v v
+-----------+ +-----------+
| Replica 1 | | Replica 2 |
+-----------+ +-----------+
^ ^
| |
Reads ReadsThe application can route:
POST /orders
-> Primary
GET /orders/123
-> ReplicaThis is useful for read-heavy workloads.
A common misconception:
Adding read replicas automatically increases write capacity.
Usually it does not.
If:
Writes = 10,000/secand all writes still go to one primary:
10k writes
|
v
+---------+
| Primary |
+---------+the primary remains the write bottleneck.
Read replicas primarily help with reads.
To scale writes, systems may eventually need techniques such as:
Consider:
Client
|
| 1. Create order
v
Primary
|
| success
v
Client
|
| 2. Immediately GET order
v
ReplicaSuppose replication has not caught up.
The replica might return:
404 Not Foundeven though the order was successfully created.
This is a classic replica-lag problem.
A user often expects:
If I successfully write something, I should immediately be able to see it.
This is called a read-your-writes expectation.
A common solution is to route a user's reads to the primary for a short period after a write.
Write
|
v
Primary
|
v
User
Immediate read
|
v
PrimaryLater:
Read
|
v
ReplicaAnother approach is to use replication positions/consistency mechanisms provided by the database.
The exact implementation is database-specific.
A stale read occurs when a replica returns older data than the current primary state.
Example:
Primary:
status = "PAID"
Replica:
status = "PENDING"This may be acceptable for some workloads.
Examples where stale data may be tolerable:
Examples where stale data can be problematic:
The acceptable staleness depends on business requirements.
A service may use a database abstraction:
Application
|
+-----------+-----------+
| |
Writes Reads
| |
v v
Primary Read Router
|
+-----------+-----------+
| |
v v
Replica 1 Replica 2The read router can:
Suppose there are three replicas:
Read Traffic
|
v
Read Balancer
/ | \
/ | \
v v v
Replica1 Replica2 Replica3Possible algorithms include:
But there is an additional database-specific concern:
replication freshness.
A healthy replica may still be too far behind.
Therefore, production systems may consider:
Health
+
Load
+
Replication Lagwhen selecting a replica.
Suppose:
Primary
|
+---- Replica 1
|
+---- Replica 2The primary crashes.
A replica may be promoted:
Replica 1
|
v
New PrimaryApplications then need to discover the new primary.
With manual failover:
Primary fails
|
v
Operator detects problem
|
v
Operator promotes replica
|
v
Application reconnectsAdvantages:
Disadvantages:
With automatic failover:
Primary fails
|
v
Failure detection
|
v
Leader election / coordination
|
v
Replica promoted
|
v
Applications redirectedThis can reduce recovery time.
However, automatic failover is difficult to implement safely.
A system must avoid promoting two nodes simultaneously.
Split-brain occurs when multiple nodes incorrectly believe they are the active primary.
Example:
Network Partition
/ \
/ \
v v
Primary A Primary B
| |
accepts accepts
writes writesNow two nodes may independently accept conflicting writes.
This can create serious data consistency problems.
Fencing is a technique for preventing an old or disconnected primary from continuing to act as the active writer.
Conceptually:
Old Primary
|
X
Fenced / prevented from writing
New Primary
|
v
Accepts writesThe exact fencing mechanism depends on the infrastructure.
The core principle is:
Before promoting a new writer, make sure the old writer cannot continue safely writing.
This is one of the most important ideas in failover design.
In automatic failover, nodes may need to agree on which node becomes leader.
Simplified:
Node A
Node B
Node C
|
v
Coordination
|
v
Leader = Node BDistributed coordination systems may use consensus protocols to make such decisions.
We will study distributed coordination in much greater depth later in the course.
For now remember:
Failover is not simply “pick another server.”
It requires coordinated ownership of the write role.
A replica can be promoted to primary.
Before promotion, consider:
Example:
Before:
Primary A
|
+---- Replica B
+---- Replica C
After:
Primary B
|
+---- Replica CReplica B becomes the new writer.
Replication can be organized in different ways.
Primary
/ \
v v
Replica ReplicaMost straightforward model.
Instead of every replica receiving directly from the primary:
Primary
|
v
Replica A
|
+----> Replica B
|
+----> Replica CThis can reduce direct replication load on the primary in some architectures.
But it introduces additional dependency and potentially more replication lag.
Multiple nodes can accept writes.
Primary A <------> Primary B
^ ^
| |
Writes WritesAdvantages:
Challenges:
Example:
User in India
|
v
Primary India
User in US
|
v
Primary USIf both update the same record at nearly the same time, the system must define what happens.
In some distributed databases there is no single leader responsible for all writes.
Instead:
+-----------+
| Node A |
+-----------+
/ \
/ \
v v
+---------+ +---------+
| Node B | | Node C |
+---------+ +---------+Clients may write to multiple replicas according to the database's consistency protocol.
Quorums can be used to determine how many nodes need to acknowledge an operation.
Leaderless systems can provide strong availability characteristics but require careful reasoning about consistency and conflict resolution.
A simplified quorum model:
N = number of replicasW = acknowledgments required for a writeR = replicas consulted for a readA common rule for overlapping read/write sets is:
R + W > NThis can help ensure that a read and write overlap on at least one replica.
However, this equation alone does not define the complete consistency behavior of a real database.
Database-specific semantics matter.
Suppose an application operates globally:
Global Users
|
+-------------+-------------+
| | |
v v v
India US Europe
| | |
v v v
DB Region A DB Region B DB Region CReplication can keep data available across regions.
Example:
India DB
|
+------> US DB
|
+------> Europe DBBenefits:
Challenges:
Suppose:
India
|
| 150 ms RTT
v
USIf every write requires a remote synchronous acknowledgment, write latency can increase significantly.
This is one reason many globally distributed systems use asynchronous replication between regions.
The exact architecture depends on the consistency and durability requirements.
RPO means:
Recovery Point Objective
It answers:
How much data loss can the business tolerate after a failure?
Example:
RPO = 0means the target is effectively no data loss.
With asynchronous replication:
Primary
|
| replication lag
v
ReplicaIf the primary fails before the replica receives the newest data, those writes may be lost during failover.
Therefore:
Replication strategy
|
v
Potential data loss
|
v
RPOReplication is closely related to disaster recovery, but replication alone is not a complete disaster-recovery strategy.
RTO means:
Recovery Time Objective
It answers:
How quickly must the system recover?
For example:
RTO = 5 minutesmeans the recovery target is five minutes.
Automatic failover can reduce RTO, but it requires:
So:
Replication
+
Failover
+
Routing
+
Recovery automation
=
High-availability designThese are not the same.
Primary
|
+----> ReplicaDesigned primarily for:
Database
|
v
Backup StorageDesigned primarily for:
A dangerous misconception is:
“We have replicas, so we don't need backups.”
That is incorrect.
If a user accidentally deletes data:
Primary
|
v
DELETE
|
v
ReplicaThe deletion may also replicate.
A backup can provide a recovery point from before the deletion.
A disaster-recovery architecture may look like:
Primary Region
+-----------+
| DB Primary|
+-----+-----+
|
Replication
|
v
DR Region
+-----------+
| DB Replica|
+-----------+If the primary region becomes unavailable:
Primary Region
X
|
v
DR Region
becomes
activeBut the DR plan must also account for:
A database replica alone does not make the entire application disaster resilient.
Replication should be monitored continuously.
Important metrics include:
Lag = current primary position - replica positionThe exact measurement depends on the database.
Monitor:
How quickly is the replica applying incoming changes?
A sudden increase in writes may cause replicas to fall behind.
Suppose:
Primary writes: 50,000 changes/sec
Replica applies: 30,000 changes/secThe backlog grows.
Eventually:
Replication Lag
|
v
IncreasingPotential responses:
A replica should not necessarily serve user traffic just because it is technically alive.
These are different.
A replica can be:
Healthy = YES
Fresh = NOFor example:
Replica:
CPU = 30%
Memory = 50%
Connection = OK
Replication lag = 60 secondsThe server is healthy from an infrastructure perspective but may be unsuitable for some consistency-sensitive reads.
This is a common system-design interview point.
Consider an e-commerce system.
Traffic:
Reads = 100,000/sec
Writes = 5,000/secA possible architecture:
Application
|
+----------+----------+
| |
Writes Reads
| |
v v
+--------+ +-------------+
|Primary | | Read Router |
+---+----+ +------+------+
| |
Replication +-----+-----+
| | | |
+------+-----+ v v v
| | R1 R2 R3
v v
R1 R2This separates write and read workloads.
Adding replicas does not eliminate every database bottleneck.
Suppose the primary has:
CPU = 95%
Disk = 90%
Writes = 50k/secAdding ten read replicas may not help.
Why?
Because the primary still handles:
Eventually, you may need:
We will study partitioning and sharding in the next chapter.
Applications often maintain separate database connection pools.
Conceptually:
Application
|
+---- Write Pool ----> Primary
|
+---- Read Pool -----> ReplicasThis prevents read traffic from consuming all primary connections.
Example pseudocode:
const db = operation === "write"
? primaryPool
: replicaPool;Real production systems usually use a more sophisticated routing layer.
When a primary changes:
Old Primary
X
New Primary
|
v
ApplicationExisting connections to the old primary may fail.
The application must be able to:
Retries must be designed carefully.
For writes, blindly retrying can create duplicate operations if the original request actually succeeded but the response was lost.
This is where idempotency becomes important.
Suppose:
Client
|
| Create Payment
v
Primary
|
| commits
|
X response lostThe client retries.
Client
|
| Create Payment again
v
PrimaryWithout idempotency protection, two payments could potentially be created.
A system may use:
Idempotency-Key: abc123to ensure repeated requests represent the same logical operation.
Replication does not solve this problem by itself.
Conflict is particularly important in:
Example:
Region A:
balance = 100
Region B:
balance = 100Two clients update the same record concurrently.
A -> 150
B -> 120What should the final value be?
Possible strategies depend on the data:
There is no universal conflict-resolution strategy.
| Property | Single Primary | Multi-Primary |
|---|---|---|
| Write coordination | Simpler | More complex |
| Conflict risk | Lower | Higher |
| Write scalability | Limited by primary | Can distribute writes |
| Operational complexity | Lower | Higher |
| Global writes | More challenging | Potentially easier |
| Consistency design | Simpler | More complicated |
The correct choice depends on workload and consistency requirements.
Primary >>> ReplicaReplica cannot keep up.
Nodes cannot communicate.
One replica becomes unavailable.
Replica cannot persist incoming changes.
Changes stop flowing.
Multiple nodes become writers.
A stale replica is promoted without considering lost changes.
Failover occurs but clients do not discover it.
They are not a substitute for backups.
Asynchronous replicas can lag.
Some reads require stronger consistency.
They generally increase read capacity, not primary write capacity.
Promoting a replica is useless if applications cannot find it.
Automatic failover must protect against multiple writers.
A technically healthy replica can still be stale.
Retries can duplicate non-idempotent operations.
Requirements:
Possible architecture:
Users
|
v
Load Balancer
|
v
Application Layer
/ \
/ \
Reads Writes
| |
v v
Read Router Primary
/ \ |
v v |
Replica Replica |
^ ^ |
+--------+--------------+
ReplicationBut payment/order reads may need stronger consistency.
Therefore:
Catalog reads
-> Replica
Payment status
-> Primary / consistency-aware read
Order immediately after creation
-> Primary or consistency-aware replicaSystem design is about matching architecture to requirements.
Design replication for a news website.
Requirements:
Think about:
A reasonable conceptual design might be:
Global Users
|
Global Routing
|
+--------+--------+
| |
Region A Region B
| |
v v
Replica Replica
^ ^
| |
+------+----------+
|
PrimaryThe exact topology depends on the database and availability requirements.
Create a small service with two conceptual database connections:
primary
replicaExpose:
POST /users
GET /users/:idRouting:
POST -> primary
GET -> replicaThen simulate replication lag.
For example:
POST /userswrites to primary.
The replica intentionally waits two seconds before receiving the change.
Then:
GET /users/123immediately after creation may return:
404After replication catches up:
GET /users/123returns:
{
"id": 123,
"name": "Anubhav"
}This exercise demonstrates why read-after-write consistency matters.
You have one PostgreSQL primary and three read replicas. Suddenly one replica falls 30 seconds behind. What do you do?
A strong reasoning process:
1. Confirm the lag.
2. Determine whether the replica is healthy.
3. Check CPU, memory, disk I/O and network.
4. Check replication/apply throughput.
5. Remove the stale replica from latency/consistency-sensitive read traffic.
6. Investigate why it fell behind.
7. Restore replication health.
8. Reintroduce it only when freshness is acceptable.Do not immediately assume the database itself is broken.
Find the bottleneck first.
Before deploying a replicated database architecture, ask:
Remember database replication using this model:
APPLICATION
|
+--------+--------+
| |
WRITE READ
| |
v v
+-------+ +----------+
|PRIMARY| | ROUTER |
+---+---+ +----+-----+
| |
Replication +---+---+
| | |
+-----+-----+ v v
| | R1 R2
v v
R1 R2Then ask five questions:
1. How is data copied?
2. How fresh are replicas?
3. How are reads routed?
4. What happens if primary fails?
5. How do we prevent multiple writers?If you can answer these five questions, you understand the core of database replication.
In this chapter, we learned:
The central idea is:
Replication creates additional copies of data so a system can improve availability, read scalability, and geographic resilience—but it introduces consistency, lag, failover, and operational challenges.
| Term | Meaning |
|---|---|
| Replication | Maintaining copies of database data |
| Primary | Main write node |
| Replica | Copy of primary data |
| Leader | Alternative term for primary |
| Follower | Alternative term for replica |
| Read Replica | Replica serving reads |
| Replication Lag | Delay between source and replica state |
| Synchronous | Primary waits for required replica acknowledgment |
| Asynchronous | Primary does not wait for replica application |
| Failover | Moving active responsibility to another node |
| Promotion | Turning a replica into a primary |
| Split-Brain | Multiple nodes incorrectly acting as writers |
| Fencing | Preventing an old writer from continuing |
| Multi-Primary | Multiple nodes accept writes |
| Leaderless | No single designated write leader |
| Quorum | Required subset of nodes for an operation |
| RPO | Acceptable recovery data loss |
| RTO | Target recovery time |
| Stale Read | Read returning older data |
| Read-After-Write | A write should be visible to subsequent reads |
Before moving on, make sure you can explain:
Next we move from copying data to splitting data.
We will learn:
Mark it as complete to track your mastery.