Replication
Adda's one database reboots at 2am and takes the whole app down with it. Replication keeps copies alive and reads fast — at the price of keeping them in sync.
The problem
It is 2 a.m. and the single Postgres server that holds everything Adda knows reboots for a kernel update — and Shuvo's phone lights up because for those minutes, nobody can log in, nobody can post, nobody can even load a feed. There is no second copy to fall back to. Indexes made lookups fast, but a fast database that is down is still down. One machine is one point of failure.
Even on a good night, that same machine is a bottleneck. Every read and every write funnels through it. As Adda grows, the reads alone — feeds, profiles, notifications — saturate it long before writes do. Ria needs more copies of the data: for survival, and for read throughput.
A first attempt
The naive backup is a nightly dump: copy the whole database to another disk at midnight. It protects Adda from total loss, but if the server dies at 11 p.m. you have lost 23 hours of posts, and restoring a large dump can take hours during which Adda is still down.
A nightly copy is a snapshot, not a live spare. What Ria actually wants is a second server that is continuously up to date — so that if the primary dies you switch to the copy in seconds, and so that reads can be served from the copy right now. That means streaming every change to the copy as it happens, which raises the hard question Shuvo keeps circling back to: how in-sync must the copy be?
The insight
Elect one leader that takes all writes, and stream its change log to one or more followers that stay (nearly) up to date. Reads can be served by any copy; the leader handles writes and hands over if it dies.
The single choice that defines the whole system is when the write is acknowledged:
- Synchronous: the leader waits for a follower to confirm before telling the client "done." No data loss on failover, but every write is as slow as the slowest follower.
- Asynchronous: the leader confirms immediately and ships changes in the background. Fast writes, but a leader that dies before shipping loses the last few writes, and followers briefly serve stale data.
How it works
The leader records every change to a log
Each write is appended to a replication log (the WAL / binlog). This ordered stream is the source of truth followers replay.
Followers replay the log in order
Each follower connects, streams the log, and applies the same changes in the same order, arriving at the same state a moment behind the leader.
Reads split across the copies
Writes go to the leader; reads fan out to followers. Because Adda's reads outnumber writes 10:1 or more, adding followers scales reads almost linearly.
Failover promotes a follower
If the leader dies, a follower is promoted to leader and traffic redirects to it. With synchronous replication no data is lost; with async, whatever had not shipped is gone.
Concrete numbers
Async replication lag is typically single-digit milliseconds within one datacenter, but can spike to seconds under write bursts or across regions (a cross-continent link adds 100–200 ms of round trip on its own). That lag is exactly the window where a follower serves stale data.
The read-scaling math is direct: one leader plus five read-replicas gives Adda roughly 6x read capacity, so a system doing 60k reads/s and 5k writes/s fits comfortably where a single node at 20k reads/s would have collapsed. Failover time depends on detection — health checks every few seconds plus promotion usually means 10–30 seconds of write downtime, unless an automated system does it faster.
When to use it
The trade-off
Synchronous replication protects your data but couples your write latency to your slowest replica — one slow follower slows every write. Asynchronous replication keeps writes fast but risks losing the most recent writes on failover and serving stale reads. A common middle ground is semi-synchronous: wait for one follower, not all.
Read-your-own-writes
Right after Fahim posts a comment, a read from a lagging follower may not show it — the comment "vanishes." Fix it by routing that user's reads to the leader (or to a follower known to be caught up) for a short window after they write. This is why replica reads need care, not just a load balancer.
Practice
Recap
- One leader takes writes; followers replay its log and serve reads.
- Synchronous = no data loss but slow writes; asynchronous = fast writes but stale reads and a small loss window on failover.
- Replication scales reads and gives you failover — it does not scale writes.
Sharding
Splitting writes across machines when one leader is not enough.
Consistency & the CAP Theorem
Why replication forces a consistency-versus-availability choice.
Redundancy and Failover
Turning replicas into automatic recovery.
In an interview
When you say "add read replicas," immediately name the two costs: replication lag (stale reads) and that writes still bottleneck on the leader. Then reach for the sync-versus-async trade and mention read-your-own-writes. Interviewers want to hear that replication buys read scale and availability, not write scale.
How is this guide?
Last updated on
Indexing
Adda's login page starts timing out, and Shuvo finds the database reading every row to find one. Finding a needle in millions by checking each straw is hopeless — an index is the book's table of contents for your data.
Sharding
Adda's data outgrows the biggest machine the cloud rents. Ria has to split it across many — sharding is that split, and choosing the key by user id is everything.