Cache Stampede and How to Prevent It: A System Design Guide

If you remember five things

The whole concept in one glance. Everything below defends these five calls.

The miss path is the danger. A cache hit is free. The cliff is concurrent duplicate misses on one hot key, all recomputing the same value at once.
Hotter means worse. Your most important data is your most dangerous, because popularity is what turns one miss into a thousand simultaneous queries.
Single-flight collapses the herd. One request recomputes behind a short-lived lock, everyone else waits and reads the freshly filled value.
Stale-while-revalidate never blocks. Serve the expired value instantly, refresh async, trade a few seconds of staleness for the guarantee that nobody waits on a recompute.
Jitter is the cheapest fix. Randomize TTLs so a batch-populated cache does not expire in lockstep. One line turns a cliff into a slope.

1The core idea#

Hold the mental model first. The cache sits in front of a slow, expensive source of truth, and the trouble lives entirely on the miss.

Your cache fronts something slow: a database, an external API, a heavy computation.
Hit: the key is present, you serve it fast and cheap.
Miss: the key is gone, so you compute the value, write it back, and return it.
A stampede is many requests missing on the same key at the same instant, each independently deciding to recompute.
One trip to the database becomes a thousand. Or ten thousand. The cache stepped aside and full production traffic landed on a box that was never sized for it.

Same animal, many names. Cache stampede, thundering herd, dogpile, cache miss storm. All one thing.

The one sentence that matters

The problem is not the miss. One miss is fine. The problem is concurrent duplicate misses on a hot key. The hotter the key, the worse the stampede, which is a cruel property: your most important data is your most dangerous.

2How it works#

Walk the classic trigger with real numbers: a plain TTL expiry on a hot key.

The setup (a flash-sale product page).

Viewed 50,000 times a second.
TTL of 60 seconds.
Database answers that product query in about 20ms.
A single instance comfortably handles about 2,000 of those per second before CPU saturates.

The moment it expires.

At second 60 the entry evicts.
Request 1 finds nothing, queries the database.
Request 2 arrives a microsecond later, also finds nothing (request 1 has not finished writing), also queries.
So does request 3, and request 4, and so on.

Now the arithmetic.

The query takes 20ms to return and write back.
At 50,000 requests per second, 20ms is a window of 1,000 requests.
So roughly 1,000 requests each launch their own identical query before the first write lands.
You just asked a box rated for 2,000 queries per second to absorb a 1,000-query spike in 20ms, on top of its normal load.
CPU pins. The 20ms query now takes 400ms under contention, which widens the window, which pulls in even more duplicates. That is the feedback loop.
One expired key, and a database that was 1% utilized is on the floor.

~1,000

Duplicate queries from one expiryConcurrent identical queries fired into a 20ms recompute window at 50,000 reads/second. This is the blast radius. Every fix in this guide exists to drive this number back down to one.

sequenceDiagram participant C1 as Client A participant C2 as Client B participant C3 as Client C participant Cache as [Cache] participant DB as [Database] Note over Cache: key expires at t0 C1->>Cache: GET product 42 Cache-->>C1: miss C2->>Cache: GET product 42 Cache-->>C2: miss C3->>Cache: GET product 42 Cache-->>C3: miss C1->>DB: query product 42 C2->>DB: query product 42 C3->>DB: query product 42 Note over DB: duplicate work, load spikes DB-->>C1: row C1->>Cache: SET product 42 DB-->>C2: row DB-->>C3: row

Three clients all miss on the same key in the gap before the first SET lands, so all three hit the database with the identical query. Notice that nothing is broken: every client is behaving correctly. The pileup is purely a timing window, and widening that window (slow DB) makes it worse.

Pause before you read on: the fix has to make only one worker recompute while the rest wait. Name the three families of techniques that do this, and the one thing they trade against each other.

Reveal the answer

Locking (single-flight), probabilistic early recomputation, and stale-while-revalidate. The axis they trade on is consistency versus protection: locking keeps data fresh but adds tail latency for the losers, while serving stale never blocks but hands some readers an out-of-date value. Naming that axis, not just the three fixes, is what separates a memorized answer from an operated one.

The fix in one line: when a key is missing, exactly one worker recomputes it while everyone else waits or serves something slightly stale. Three families of technique get you there.

Locking (single-flight)#

Before recomputing, a request tries to grab a short-lived lock on the key, usually a SET key NX EX 10 in Redis.
Winner does the work, writes the value, releases.
Losers wait briefly and retry the cache read, by which point the winner has usually filled it.
In the flash-sale example, this collapses 1,000 concurrent misses into exactly one real database query. The other 999 read the freshly filled cache within a few milliseconds.
Many request-coalescing libraries do this in-process too, so a single server deduplicates its own concurrent callers before touching Redis at all.

The single-flight gate. The only path to the database is through the lock, so one winner does the real work. The losers loop back through the cache read (F to B), which is where they pick up the value the winner just wrote. The retry loop is the part to watch: if too many losers spin, you have moved the pressure, not removed it.

Early recomputation (probabilistic early expiry)#

Refresh the key a little before it dies, on only one request, so the crowd never sees a hard miss.
Store the value with its computation cost and a logical expiry.
Each read rolls the dice: the closer you are to expiry, the higher the chance this reader refreshes early in the background while still serving the current value.
The math spreads refreshes out so one request volunteers before anyone sees a miss.
This is the approach from the well-known "optimal probabilistic cache stampede prevention" work. It is elegant because there is no lock and no hard miss.

Stale-while-revalidate#

Serve the expired value anyway, and kick off an async refresh.
The user gets a slightly old answer in single-digit milliseconds. The fresh value lands a moment later for the next reader.
HTTP caches and CDNs expose this directly through the stale-while-revalidate cache-control directive.
It trades a bounded amount of staleness for the guarantee that nobody ever blocks on a recompute.

3Trade-offs#

None of these is free. The interviewer wants to hear what you give up.

Technique	What you gain	What you pay	Best when
Distributed lock	One DB query per key, strong dedup	Added latency for waiters, lock infra, deadlock and expiry edge cases	Recompute is very expensive and you need freshness
Probabilistic early refresh	No hard misses, no lock, smooth load	Slightly more total recomputes, tuning the beta factor	High-read hot keys where a tiny bit of early work is cheap
Stale-while-revalidate	Fast responses always, simple	Serves stale data for a window	Staleness of a few seconds is acceptable
Longer TTL	Fewer expiries, fewer stampedes	More staleness, harder invalidation	Data changes rarely
No expiry plus active refresh	Zero expiry-driven stampedes	You must build a refresh pipeline and handle failures	A known, bounded set of hot keys

The big axis: consistency versus protection.

Locking keeps data fresh but adds a serialization point and tail latency for everyone who lost the race.
Stale-while-revalidate is the opposite: never blocks, but some users read a value a few seconds out of date.
A product price during a sale: stale may be unacceptable, because showing an old price you cannot honor is a real problem.
A trending-posts list: nobody will ever notice or care if it is 5 seconds behind.

The second axis: complexity.

A longer TTL with jitter is ten minutes of work.
A probabilistic early-refresh layer with per-key cost tracking is real engineering.
Do not reach for the fancy solution when raising the TTL and adding randomness would have solved it.
I once watched a team spend a sprint on a distributed-lock refactor when the real bug was that every cache key had the same 5-minute TTL, so they all expired in lockstep every 300 seconds like clockwork.

Synchronized expiry deserves its own name.

Populate a cache in a batch job and every key gets the same TTL, so they all die together.
The cheapest fix in this whole guide is jitter: instead of TTL = 300, use TTL = 300 + random(0, 60).
That one line smears expiry across a 60-second window and turns a cliff into a gentle slope.

Always add jitter. It is one line, and it is the fix most teams skip until the pager goes off.

4Where it is used#

Every serious caching layer has some answer to this, whether the team named it or not.

Facebook memcache leases (the Scaling Memcache paper): the cache hands exactly one client a token to recompute a missing key, and everyone else waits or serves stale. A lock by another name, built into the cache tier itself.
CDN request collapsing: 100,000 users hit the same uncached video segment, the edge forwards one request to origin and fans the single response out to all of them. Varnish, Nginx, and the major CDN providers all do this. Single-flight at the edge: the difference between origin seeing 1 request or 100,000.
Go's singleflight package: the canonical in-process version. Concurrent calls for the same key share one execution and one result. I reach for it constantly for things like "look up this user's permissions," where a burst for one user should collapse into one backend call.

When you should design for it on purpose. Reach for stampede protection whenever you have:

(a) an expensive recompute,
(b) hot keys,
(c) TTL-based expiry.
Flash sales, viral content, a daily leaderboard that regenerates at midnight, an auth endpoint validating the same token thousands of times a second.
If any two of those three are true, bring it up before the interviewer has to.

5Common pitfalls and gotchas#

Forgetting the empty result.

A missing database row means a naive cache stores nothing, so every request for that key is a permanent miss that hammers the database forever.
This is cache penetration, a cousin of stampede.
Fix: cache the negative result (a tombstone) with a short TTL, or put a Bloom filter in front.

Locks that never release.

If the worker holding the recompute lock crashes and the lock has no expiry, every other request waits forever.
Always set an expiry, longer than your worst-case recompute but short enough that a crash does not freeze the key for minutes.
If recompute is normally 20ms and worst case is 2 seconds, a 10-second lock expiry is a sane starting point.

The waiters still pile up.

A lock stops duplicate database queries, but 1,000 requests spinning in a retry loop just moves the pressure from the database to your app servers and Redis.
Cap the wait, add backoff, and consider serving stale to waiters instead of making them block.

Jittering the wrong thing.

Jitter on the TTL helps.
Jitter does nothing if the traffic itself is synchronized, for example a cron that fires on every server at the same instant.
Look at what is actually synchronized.

Testing at low traffic.

Stampedes are a concurrency bug. They do not show up when you click around in staging.
You need a load test that drives real concurrent reads through a key expiry to see the spike.
If you cannot reproduce it under load, you have not verified your fix.

Assuming the cache tier is the only herd.

The same pattern shows up one layer down: a database query cache or materialized view that expires can stampede too.
The principle is fractal.

Red flag that fails you here

Treating the cache as pure upside. If you describe a cache and never say what happens on a miss under load, you have told the interviewer you have read about caches but never watched one take down a database. The miss path is the whole interview.

6How it shows up in interviews#

When I interview and we reach caching, I am listening for one thing: does the candidate treat the cache as a magic box, or as a system with failure modes? Mentioning stampede unprompted is a strong signal, because it means you have operated a cache and felt the pain.

What interviewers probe (and what they are really testing).

"What happens when a popular key expires?" The direct question. "It gets recomputed" misses the whole point.
"How do you keep one slow query from becoming ten thousand?" They want single-flight or locking.
"This data can be a few seconds stale, does that change your design?" They are testing whether you can relax consistency to buy availability. Lean into stale-while-revalidate.
"How would you prevent all your keys from expiring at once?" Jitter. Say the word.

From the interviewer's chair

The thing that separates senior from staff here is knowing when not to solve it. If the recompute is cheap and the key is not hot, a stampede is a non-event, and a distributed lock is just new failure modes for no benefit. Saying out loud "for these keys I would do nothing, the math does not justify it" is often the most impressive thing in the whole answer.

Weak answer, side by side with strong.

Downgrades you

The read-about-it answer

What it says: "I would add a Redis cache in front of the database with a 60-second TTL. That cuts database load a lot."
Miss path: Never mentioned.
Blast radius: No number.
Trade-off: Treats the cache as pure upside.

Earns the room

The operated-it answer

What it says: "60-second TTL plus jitter so keys do not expire together. Single-flight on hot keys so one request recomputes on a miss."
Miss path: Named as the danger, with the feedback loop.
Blast radius: "A single 20ms query becomes a thousand simultaneous queries."
Trade-off: Stale-while-revalidate if seconds of staleness are fine, negative caching for missing keys.

Follow-ups the interviewer will ask (and your answers)

"A popular key just expired. Walk me through what happens."

On a hot key, the requests that arrive during the recompute window all miss, because the first worker has not written back yet. At 50,000 reads/second with a 20ms query, that window holds about 1,000 requests, so one query becomes a thousand identical ones. The database pins, queries slow down, the window widens, and more duplicates pile in. One expired key takes down a box that was 1% utilized.

"How do you keep one slow query from becoming ten thousand?"

Single-flight. The first request to miss grabs a short-lived lock (SET key NX EX 10), does the recompute, and writes back. Everyone else waits briefly and retries the cache read, picking up the value the winner just wrote. A thousand concurrent misses collapse into one real query. In-process request coalescing dedups a single server's own callers before Redis even sees them.

"This data can be a few seconds stale. Does that change your design?"

Yes, and it lets me pick the cheaper, calmer fix. Stale-while-revalidate: serve the expired value in single-digit milliseconds and refresh async in the background, so no reader ever blocks on a recompute. The only cost is a bounded staleness window, which is free for something like a trending list and unacceptable for a live sale price.

"How do you stop all your keys from expiring at once?"

Jitter the TTL. A batch job that populates the cache gives every key the same TTL, so they all die together on a cliff. Replacing TTL = 300 with TTL = 300 + random(0, 60) smears expiry across a 60-second window. One line, and it is the fix most teams skip until the pager goes off.

"When would you do nothing at all?"

When the recompute is cheap and the key is not hot, a stampede is a non-event. Adding a distributed lock there just buys new failure modes (stuck locks, waiter pileups, extra latency) for no benefit. Knowing when the math does not justify a fix is the staff-level move, not reaching for the fanciest technique.

What you can now explain in the room

Why the miss path, not the hit, is where a cache takes down a database, and why hot keys make it worse.
How to put a number on the blast radius: 20ms recompute at 50,000 reads/second is roughly 1,000 duplicate queries.
How single-flight collapses a thousand concurrent misses into one real query, in-process and in Redis.
When to pick stale-while-revalidate over locking, framed as consistency versus protection.
Why jitter is the cheapest fix for synchronized expiry, and when jitter does nothing.
The common failure modes: cache penetration, stuck locks, waiter pileups, and low-traffic testing hiding the bug.

For a full walkthrough that puts these pieces together, see the related System Design Question on designing a read-heavy product catalog under flash-sale load.

← Previous

Cache Invalidation

Caching Strategies

Get a prep plan built around your Cache Stampede and How to Prevent It interview

Tell us the role and your interview date. We will send back a dated, day by day plan. Free.

Thanks. Check your inbox, your Cache Stampede and How to Prevent It prep plan is on the way.