Caching Fundamentals
Adda's database is melting under a flood of identical reads for the same hot posts. Ria and Tanvir stop re-answering the same question — they answer it once and remember the answer.
The problem
The CDN took the images off Adda's back, but a new graph turns red. Adda's home screen shows a "top 10 trending posts" list — the same list for everyone, computed by a heavy database query that joins several tables. At peak, tens of thousands of users hit that screen every second, and Tanvir watches the database run that identical expensive query tens of thousands of times a second. It's on fire.
The maddening part, as Ria puts it: the answer barely changes minute to minute. Adda is paying full price to recompute the exact same trending list over and over, for users who would happily accept a value a few seconds old. There has to be a way to do the work once and reuse it.
A first attempt
The instinct is to make the database faster — add read replicas, tune indexes, buy a bigger box.
That helps, but it treats the symptom. Adda is still executing the same query millions of times; the pain is just spread across more expensive machines. Replicas cost money, add lag, and still cap out. The real waste is redundant computation: recomputing an answer you already knew. Scaling the database to absorb repeated identical reads is like re-cooking a meal for every person who asks what's for dinner. Compute it once; remember it.
The insight
Store the result of expensive work in fast storage close to the reader, and serve future identical requests from there instead of redoing the work.
That store is a cache — usually in-memory (RAM), so reads are microseconds instead of the milliseconds a database query costs. The bet is locality: the same trending list gets requested far more often than it changes. When a request finds its answer in the cache, that is a hit (fast, cheap). When it does not, that is a miss — Adda does the real work, then stores the result so the next request hits. A high hit ratio means the expensive database barely runs.
How it works
Ria and Tanvir put a cache in front of the trending query:
Check the cache first
Every read looks in the cache before touching the database. This is the cache-aside pattern: on a hit, return the cached value and you are done in microseconds.
On a miss, compute and store
If the value is not cached, run the real query, return the result to the user, and write it into the cache with a TTL. The next identical request is now a hit. The first user pays; everyone after rides free.
Evict when full
Cache memory is small and precious, so it cannot hold everything. When full, an eviction policy decides what to drop — usually LRU (least recently used), betting that recently-used data will be used again.
Expire and invalidate
Each entry has a TTL so stale data self-destructs after a while. When the underlying data changes before expiry, you must invalidate — delete or update the cached copy — or readers keep seeing the old value. This is the hard part.
The cache-aside read path:
Where caches live — Adda can stack several:
- Client / browser — cache responses locally; zero network cost.
- CDN / edge — cache static responses near the user (the previous lesson).
- Application — a shared in-memory store (Redis, Memcached) for query results, sessions.
- Database — its own buffer pool caching hot pages in RAM.
The numbers
The trending list, before and after Adda's cache:
- Speed gap: RAM read ~100 ns vs SSD ~100 µs vs a real DB query ~1–10 ms — a cache is 10,000×+ faster than recomputing.
- Hit ratio is everything. At a 95% hit ratio, only 5% of reads reach the database — a 20× load reduction. Push it to 99% and the DB sees 1 in 100.
- The 5% still bites: average latency with 95% hits and a 10 ms miss ≈
0.95×0.1 ms + 0.05×10 ms ≈ 0.6 ms— dominated entirely by the misses, which is why chasing hit ratio pays off. - TTL choice: the trending list → ~30–60 s (staleness is fine). A user's account balance → seconds or write-through only (staleness is dangerous).
There are only two hard things
"There are only two hard things in computer science: cache invalidation and naming things." Caching is easy to add and hard to keep correct — the instant cached data can diverge from the source of truth, you are choosing between serving stale data and the complexity of keeping them in sync. Every cache is a bet that slightly-stale is acceptable. Know your staleness budget before you cache.
Write strategies decide your staleness
Cache-aside (lazy) is simplest but can serve stale data until TTL/invalidation. Write-through updates cache and DB together on every write — always fresh, slower writes. Write-back writes to cache first and DB later — fast writes, but a crash can lose data. Pick by how much staleness (and risk) the data tolerates.
Practice
Recap
- A cache stores expensive results in fast memory close to the reader, trading a little staleness for enormous speed and backend offload — Adda's fix for the hammered trending query.
- Hit ratio governs everything: 95% hits means the database sees 1 in 20 reads, and latency is dominated by the remaining misses.
- The hard part is invalidation — every cache is a bet that slightly-stale data is acceptable, and your write strategy (aside / through / back) sets how stale it can get.
Content Delivery Networks
Caching applied geographically at the network edge.
DNS: The Internet's Phone Book
TTLs and cache hits, seen in the wild.
HTTP & APIs
The Cache-Control headers that drive HTTP caching.
In an interview
Reach for caching the moment you spot repeated reads of rarely-changing data, and state the hit ratio you expect and why. Then get ahead of the follow-up: name your invalidation strategy and your staleness budget out loud. Saying "this can be stale for 60 seconds, so cache-aside with a 60s TTL" shows you understand caching is a correctness trade, not a free speedup.
How is this guide?
Last updated on
Content Delivery Networks
Adda's image-heavy posts crawl for users far from Dhaka. Ria and Tanvir can't move Dhaka closer to Sydney — so they move the content instead, with copies at the edge.
SQL vs NoSQL
Adda's data model is finally firming up, and Ria has to answer the question she has dodged for months — which database? Shuvo joins to make her answer it on purpose.