A cache is a small, fast store that sits in front of a slow one. If the data you want is already in the cache, you skip the slow database. That can turn a 20 ms (millisecond) database query into a 0.5 ms memory lookup — about 40 times faster. Caching is the most powerful tool you have for making a system feel fast. It also comes with the most famous hard problem in the field: knowing when to throw old cached data away (called invalidation).
The animation on the right walks through all three caching strategies one at a time — cache-aside (look in the cache, miss, go to the database, then save the result), write-through, and write-back — so you can watch how each one trades off freshness, speed, and safety.
Hit vs. miss
Cache hit / miss
A hit means the data you asked for was already in the cache, so you got it fast. A miss means it wasn't there, so you had to go to the slow database — and you usually save the result in the cache so the next request is a hit.
The number to watch is your hit ratio — the share of reads the cache can answer by itself. A 95% hit ratio means the database only sees 1 read out of every 20. Pushing that ratio from 90% to 99% cuts database load by 90%. That is the big, non-obvious win of tuning a cache: a small jump in hit ratio is a huge drop in database work.
Read strategies
These describe how you read data when a cache is involved.
Cache-aside (lazy loading)
The app checks the cache itself. On a miss, the app reads the database and then copies the result into the cache. This is simple and tough — if the cache breaks, reads still work because the app just goes to the database. It is the most common pattern. The one downside: the very first read of any key is always a miss, because nothing is in the cache until someone asks for it once. We call this a "cold" cache.
1. App reads from cache
2. Miss → App reads from DB
3. App writes result to cache
4. Next read → cache hit
Read-through
Here the cache itself goes to the database on a miss, and it does this quietly behind the scenes. The app only ever talks to the cache. This is handy when you want all the database-fetching logic in one place. It has the same cold-start problem: the first read of a key is still a miss.
Write strategies
These describe how you write (save) data when a cache is involved.
Write-through
Every write goes to the cache and the database at the same time, and both must finish before the write is done. The cache is always up to date, but each write costs you two saves instead of one.
Write-back (write-behind)
Every write goes to the cache right away and returns immediately. The database is updated later, in the background. Writes feel very fast, but if the cache dies before it copies the data to the database, that data is lost.
Write-around
Writes go straight to the database and skip the cache. The next read of that key will be a miss. This is best when data is written once and almost never read again (for example, logs or analytics).
| Strategy | Write speed | Cache freshness | Data safety | Best for |
|---|---|---|---|---|
| Write-through | Slower | Always fresh | Safe | Read-heavy, must stay correct |
| Write-back | Very fast | Slightly stale | Risk of loss | Write-heavy, can survive brief data loss |
| Write-around | Fast | Stale on next read | Safe | Written once, rarely read |
| Cache-aside | Normal | Catches up over time | Safe | Most general-purpose APIs |
Cache invalidation
Phil Karlton famously said: "There are only two hard things in Computer Science: cache invalidation and naming things." Invalidation means deciding when a cached copy is too old to trust and removing it. Here is why that is hard, and three common ways to handle it:
- TTL (Time-To-Live): Each entry expires by itself after N seconds. Simple and self-cleaning, but the data can be a little out of date until it expires. (Fine for a product list; not fine for a bank balance.)
- Delete on write: When the database record changes, you immediately delete or update the matching cache entry. This is correct, but it ties your write code to your cache.
- Event-driven: When something changes, you publish a "user updated" message. A cache service listens for it and clears the matching entry. This keeps things loosely connected, but it needs extra infrastructure to run.
When asked "how do you keep the cache consistent?", name one concrete approach: a TTL (expire after N seconds — simple, allows a little staleness) or delete-on-write (correct, but more moving parts). The key signal is saying that some staleness is a deliberate trade-off you chose, not a bug.
Eviction policies
When the cache fills up, it has to throw something out to make room. Which entry it removes matters:
LRU (Least Recently Used)
Throw out the key that hasn't been touched for the longest time. This is the most common choice, because data used recently is likely to be used again soon.
LFU (Least Frequently Used)
Throw out the key that has been used the fewest times overall. This beats LRU when you have a steady set of "hot" keys that are always popular.
TTL-based eviction
Every key has an expiry time, and the cache removes it when that time is up — no matter how often it was used.
Most real systems use LRU with TTLs as a safety net. Redis supports both and lets you set a memory limit and an eviction policy for each cache.
When should you cache at all?
Caching isn't free. It adds another piece to run, a fresh consistency problem, and a new way for things to break. The simple rule: cache data that is read often but changed rarely. A user's profile, read on every page but edited once a month, is a great fit. A counter that goes up on every single request is not.
The other half of the decision is what you can afford to lose. A cache lives in memory only — restart the cache server and everything in it is gone. So a cache is a bad place to keep the only copy of anything.
Never treat the cache as your source of truth. Cached data is volatile — if a cache server restarts, everything in memory is lost. Important data must always be saved to a durable store (your database). The cache speeds up access to the truth; it is never the truth itself.
When you propose a cache, say out loud what you're caching and why: "profiles are read on every request but updated rarely, so they're a great cache candidate — and since the cache is volatile, the database stays the source of truth." Naming both the read/write balance and the durability boundary in one sentence is the senior signal.
Sizing the cache tier (back-of-the-envelope)
When an interviewer asks "how much RAM does your cache need?", don't try to cache everything. Use the 80/20 rule: about 20% of your keys serve about 80% of the traffic. Size the cache to fit that hot 20% (your "working set"), not the whole dataset.
Say you have 50M cacheable items, each about 1 KB (a serialized user profile). Work out the total first, then the hot subset:
Total dataset = 50M items × 1 KB = 50 GB
Hot set (top 20%) = 50 GB × 0.20 = 10 GB
+ overhead (~30% for Redis metadata/frag) ≈ 13 GB
Round up: one Redis box with 16–32 GB of RAM holds the entire hot set — no need to split it across machines yet. Now check what that saves on the database. Say reads come in at 20,000 QPS (queries per second):
At 95% hit ratio → DB sees 5% of reads = 0.05 × 20,000 = 1,000 QPS
At 99% hit ratio → DB sees 1% of reads = 0.01 × 20,000 = 200 QPS
Those last 4 points of hit ratio cut database read load by 5× (1,000 → 200 QPS). That is the non-obvious payoff: you size for the hot 20% of data, and a high hit ratio shrinks the database load to almost nothing — turning a whole fleet of read replicas into a single primary that is barely working.
| Quantity | Value | How |
|---|---|---|
| Total dataset | 50 GB | 50M × 1 KB |
| Cached hot set | ~13 GB | 20% + 30% overhead |
| Boxes needed | 1 | 13 GB fits one 16–32 GB node |
| DB reads @ 95% hit | 1,000 QPS | 5% of 20K |
| DB reads @ 99% hit | 200 QPS | 1% of 20K |
When sizing a cache, don't just multiply total items by item size and provision that — that's the junior move. Say "I'll cache the hot ~20% of keys, so 10 GB not 50 GB, plus ~30% Redis overhead — that fits one box, and at a 95% hit ratio the database only sees 5% of reads." Sizing to the working set instead of the full dataset, and linking hit ratio straight to database QPS, is the senior signal.
A single cache is a single point of failure
One cache server holding your entire hot dataset is a single point of failure (SPOF) — if it dies, everything breaks. Every request that used to hit the cache now slams the database, exactly when you can least afford it. Two standard fixes:
- Run several cache servers, ideally across data centers. Spreading the cache across many nodes (and across regions) removes the one weak spot, the same way database replicas remove a single-database weak spot.
- Give the cache extra memory headroom. Size it with a buffer above what you expect to use, so a traffic spike or steady growth doesn't immediately start evicting your hot data.
| Concern | Single cache server | Mitigation |
|---|---|---|
| Availability | One node dies → cache tier down → DB overwhelmed | Several cache servers across data centers |
| Capacity | Fills up → evicts hot keys, hit ratio collapses | Give it extra memory as a growth buffer |
| Blast radius | All traffic funnels to one node | Spread keys across many nodes |
If you sketch a single Redis box in front of the database, get ahead of the follow-up: "that single cache is a SPOF, so in production I'd run several cache servers across data centers and over-provision memory as a growth buffer." Spotting your own single point of failure before the interviewer does reads as senior.
Cross-region consistency is the hard part
Keeping the database and the cache in sync is already tricky. The write to the database and the matching update to the cache are not one single all-or-nothing operation — so there's always a tiny window where one has happened and the other hasn't. A crash, a retry, or a reordering inside that window leaves the cache holding stale data.
This gets much harder across regions. Now you have many cache tiers in many data centers, each racing the same database writes, with replication lag piled on top. There is no clean way to wrap every regional cache and the database in one transaction, so some inconsistency always creeps in. The real engineering work is bounding how stale and for how long — not making it disappear.
The classic real-world write-up is Facebook's paper "Scaling Memcache at Facebook." It's worth naming in an interview as the reference for how a huge system handles cache consistency across many regions — including tricks like leases and per-key invalidation streams.
When asked about caching across regions, be honest that perfect consistency isn't possible: "the database update and the cache update aren't one transaction, so multi-region cache consistency is a known hard problem — Facebook's 'Scaling Memcache at Facebook' paper is the canonical treatment." Naming the limit and a real source beats hand-waving a guarantee you can't deliver.
The thundering herd problem
When a popular cache key expires, lots of requests — sometimes thousands — all miss at the same moment and all rush to the database for the same data at once. The database gets a sudden flood of identical queries. This is called a "thundering herd."
Solutions:
- Lock on miss: The first request grabs a lock, fetches from the database, and fills the cache. The other requests wait. Once the lock is released, they all read from the cache.
- Refresh early: Re-build the cache entry a little before its TTL runs out, so the stampede never starts.
- Jitter the TTL: Instead of giving every key the same TTL of 300s, use
TTL = 270 + random(0, 60). Keys then expire at slightly different times, spreading the misses out.
Redis vs. Memcached
Both are in-memory key-value stores used as caches. The main differences:
| Redis | Memcached | |
|---|---|---|
| Data types | Strings, lists, sets, sorted sets, hashes, streams | Strings only |
| Persistence | RDB snapshots + AOF log (optional) | None |
| Replication | Built-in primary/replica + Cluster mode | Client-side sharding |
| Lua scripting | Yes | No |
| Best for | General cache + pub/sub + rate limiting + leaderboards | Pure high-throughput caching |
Default to Redis. Its richer data types unlock jobs beyond plain caching: rate limiting (sorted sets), leaderboards (sorted sets), pub/sub messaging, and distributed locks. Memcached only wins when you need the absolute maximum throughput out of many threads.
Where caches live
Caching happens in layers — the same request can be cached at several levels along the way:
- Client cache — the browser saves responses based on
Cache-Controlheaders - CDN cache — edge nodes around the world cache static files (Chapter 7)
- Application cache — in-process memory (like a Java
HashMap) for tiny, hot datasets - Distributed cache — Redis/Memcached shared by all application servers
- Database cache — the database engine's own buffer (the InnoDB buffer pool)
Each layer is faster and smaller than the one below it. The goal is to answer as many requests as possible at the highest (cheapest) layer.
Cache warming: When you deploy a new service or restart a cache, it starts cold — every request is a miss. Pre-filling ("warming") the cache by replaying recent popular queries before sending real traffic to it reduces the slow spike at startup.
Practice
Answer these to check you understood caching.
1. What is a cache hit ratio?
2. In cache-aside, what happens on a miss?
3. Why is write-back fast but risky?
4. Why should you never treat the cache as your source of truth?
5. Which fix helps with the thundering herd problem?