Caching Strategies

1The core idea#

A cache is a small, fast copy of data that sits closer to whoever needs it. Speed is the entire point:

• A relational database reading from disk answers in 5 to 20 ms.
• Redis in memory answers in roughly 0.2 to 0.5 ms, call it 50x faster.
• A local in-process map answers in a few hundred nanoseconds, another 1000x on top of that.
• The game is simple: keep the hot data in the fast place, and keep it correct enough.

The mental model I give juniors: a cache is a second source of truth that is allowed to lie, briefly.

• The database is the truth.
• The cache is a convenient, faster, slightly stale version of it.
• A cache is not a feature you bolt on. It is a correctness trade you opt into on purpose.

The payoff tracks the read-to-write ratio, so find it first.

• A product catalog read 10,000 times for every update is a perfect cache target.
• A bank ledger where every read must be exact and writes are frequent is a terrible one.
• The ratio tells you most of the answer before you even pick a strategy.

2How it works#

The strategies split into read patterns and write patterns. You pick one of each and combine them. That framing alone separates you from candidates who recite five names as if they were interchangeable.

Cache-aside (lazy loading). The default, and the one to reach for unless you have a reason not to.

• The application code owns the logic.
• On a read, check the cache first. Hit? Return it. Miss? Go to the database, put the result in the cache, then return it.
• The cache only fills with data someone actually asked for, which keeps it lean.

Read-through. Same read flow, but the cache library handles the miss for you.

• Your code just asks the cache; the cache knows how to load from the database on a miss.
• Cleaner code, but you give up control and you are tied to a cache that supports it.
• Functionally close to cache-aside. The only difference is who writes the "on miss, go fetch" logic: you or the library.

Now the writes, which is where most people get sloppy.

Write-through. Every write goes to the cache and the database together, synchronously, before you acknowledge the client.

• The cache is always consistent with the database.
• The cost is write latency: you pay for both hops on every write.
• If the database write is 8 ms and the cache write is 0.3 ms, every write now costs about 8.3 ms instead of 8.
• Good when you read what you just wrote and you cannot tolerate staleness.

Write-back (write-behind). Write to the cache, acknowledge immediately, and flush to the database later in batches.

• Writes feel instant, around 0.3 ms, and you can coalesce many updates into fewer database hits.
• The danger is obvious: if the cache node dies before the flush, you lose data.
• A real tool, used in high-write systems like metrics ingestion.
• Never propose it in an interview without saying "and here is how we survive a crash" in the same breath: append-only log, replication, or a bounded flush window.

Write-around. Writes skip the cache and go straight to the database. The cache only fills on later reads (via cache-aside).

• This avoids polluting the cache with data that is written but rarely read.
• Think of a logging table: lots of writes, few reads.

Here is the sequence for the combo I use most, cache-aside reads with a write that invalidates:

3A worked example with real numbers#

Interviewers love watching you do the arithmetic, so make it concrete. Say we run a product listing service:

• 50,000 read requests per second at peak.
• A product record changes maybe 5 times per second across the whole catalog.
• That is a 10,000:1 read-to-write ratio, a textbook cache target.

Without a cache.

• All 50,000 reads hit the database.
• At 8 ms per read, one Postgres primary doing maybe 8,000 reads/sec is swamped at 50,000.
• You would need 6 to 7 read replicas just to survive, and p99 latency would still be ugly under load.

Now add Redis with cache-aside and a 300-second TTL. Suppose the working set of hot products gives us a 95% hit rate (easy to hit for catalog data):

• Cache hits: 50,000 x 0.95 = 47,500 reads/sec served from Redis at ~0.3 ms.
• Cache misses: 50,000 x 0.05 = 2,500 reads/sec hit the database at ~8 ms.
• Database load drops from 50,000 to 2,500 reads/sec, a 20x reduction. One primary with no replicas handles that comfortably.
• Average read latency: 0.95 x 0.3 + 0.05 x 8 = 0.285 + 0.4 = 0.685 ms, versus 8 ms before, roughly 12x faster on average.

The staleness cost of that 300-second TTL.

• A price change can be up to 5 minutes stale unless the write path deletes the key.
• So I delete on write and keep the TTL as a safety net.
• That pulls staleness down to the few milliseconds between the database write and the cache delete.

4Trade-offs#

Here is how the main strategies stack up on the things that matter.

The two write strategies people confuse most are write-through and write-back, so pin them side by side:

The trade-offs nobody says out loud:

Consistency vs speed.

• Every cache introduces a window where the cache and the database disagree.
• Write-through shrinks that window at the cost of write latency.
• Cache-aside with TTL accepts a bounded staleness window in exchange for simpler, faster writes.
• There is no "no staleness and no latency cost" option. If someone claims one, they have hidden the cost somewhere.

Memory vs hit rate.

• A bigger cache holds more data and hits more often, but memory costs money and eviction gets complicated.
• Going from a 95% to a 99% hit rate might mean holding 4x the working set: the difference between a 13 GB Redis node and a 55 GB one.
• The eviction policy decides what to drop when you are full. LRU is the safe default.
• LFU is better when you have a stable set of very hot keys that should not get pushed out by a burst of one-off reads.

TTL length.

• Short TTLs mean fresher data and more database load. Long TTLs mean less load and more staleness.
• I usually start around 60 seconds for a product page and tune from there with real hit-rate numbers.
• Do not pick a TTL in an interview without saying what you are trading.

5Where it is used#

Caching shows up at every layer, and strong candidates name the layer:

• Browser and CDN. Static assets, images, and whole API responses cached at the edge with cache-control headers. A Cloudflare or Fastly node serving a cached response never touches your origin. The cheapest, highest-leverage cache you have.
• Application-level (Redis or Memcached). The classic shared cache: session data, computed feeds, rate-limit counters, hot database rows. Redis also gives you atomic ops and pub/sub, which Memcached does not.
• In-process (local) cache. A map inside the service process, like Guava on the JVM or a simple LRU in Node. Sub-microsecond reads, but each instance has its own copy, so you get consistency drift across instances. Great for config and feature flags that change rarely.
• Database and query cache. The database's own buffer pool, plus materialized views for expensive aggregations.

Concrete example: a product listing page, where one page needs two different decisions.

• Product metadata (name, price, description) changes maybe a few times a day but gets read millions of times. Cache-aside it in Redis with a 5-minute TTL and delete the key on any admin edit.
• The inventory count changes every second. Do not cache it the same way. Either skip the cache for it or use a 2-second TTL and accept that "in stock" might be a couple seconds stale.
• Same page, two decisions, because the read-to-write ratio is different for each field. Saying that out loud is the move that lands.

6Common pitfalls and gotchas#

Cache stampede (thundering herd).

• A popular key expires and 5,000 requests all miss at once and all hit the database together, which falls over.
• Fix one: add short random jitter to TTLs so keys do not expire in lockstep (300 seconds plus or minus 30).
• Fix two: use a lock so only one request rebuilds the key while the others wait.
• Fix three: refresh hot keys in the background before they expire.
• Mentioning this unprompted is a strong signal in an interview.

Updating the cache on write instead of deleting.

• If two writes race, you can leave a stale value in the cache forever.
• Deleting the key is safer: worst case, the next read takes a miss and reloads the truth from the database.
• I delete, I do not update, unless I have a specific reason.

Thinking the cache fixed your consistency.

• It did not. You chose a staleness window.
• Be able to state it: "reads can be up to 60 seconds stale."
• If you cannot say the number, you do not understand your own design.

Caching things you should not.

• Per-user private data in a shared cache without namespacing the key is how you serve user A's data to user B. Always scope keys, like user:1234:cart.
• Do not cache data that is cheaper to recompute than to store.

Ignoring the cold-start or cold-cache problem.

• Right after a deploy or a Redis restart, the cache is empty and every request is a miss.
• In the worked example that means the database jumps from 2,500 to 50,000 reads/sec in an instant, 20x, and falls over.
• Warm critical keys on startup, or ramp traffic, or the first minute after deploy becomes an outage.

No plan for when the cache is down.

• If Redis goes away, does your service fail or fall back to the database?
• If it falls back, can the database handle 100% of traffic suddenly? In our example that is 50,000 reads/sec against a primary sized for 2,500. Usually not.
• Have an answer before the interviewer asks.

7How it shows up in interviews#

Interviewers use caching to separate people who memorized "add Redis" from people who have run systems. Here is what they probe and what good answers sound like.

"How do you keep the cache consistent with the database?"

• Weak: "The cache updates automatically." (It does not, and this reveals you have never wired one up.)
• Strong: "I would use cache-aside and delete the key on write, so the next read reloads from the database. That gives me a staleness window bounded by the TTL, say 60 seconds, which is fine for a product page but not for inventory. The actual inconsistency window is just the few milliseconds between my database write and my cache delete."

"What happens when a hot key expires under load?"

• Weak: "It just reloads." (Technically true, operationally a disaster.)
• Strong: Names the stampede, puts a number on it ("5,000 concurrent misses on one key all hitting Postgres at once"), then gives a fix: TTL jitter, a rebuild lock, or background refresh.

"The cache goes down. What happens?"

• Weak: Hasn't thought about it.
• Strong: "Reads fall back to the database, but I need to confirm the database can take full read load. In my sizing it is handling 2,500 reads/sec and would suddenly see 50,000, so it cannot. I would add request coalescing and a local in-process cache as a second layer so we do not melt the primary."

"Why not just cache everything with a long TTL?"

• Weak: "Sure, that is faster." (Misses the staleness and memory cost entirely.)
• Strong: Connects it back to the read-to-write ratio and the staleness tolerance of each piece of data, and points out memory cost and eviction. The move that lands best is treating different fields on the same page differently: 5-minute TTL on the price, 2-second TTL on the stock count.

Next up, try the System Design Question walkthrough on designing a URL shortener, where caching the hot short-codes is the difference between a fast service and a database on fire.