Content Delivery Networks (CDN)

1The core idea#

Keep copies of your content close to the people who want it. That is the whole mental model.

• A CDN is a fleet of caching servers spread across many physical locations, called edge locations or points of presence (PoPs).
• Big providers run 300 or more PoPs across 100-plus cities.
• A request gets served from the PoP nearest the user instead of from your single origin far away.
• Your origin (an S3 bucket, a web server, an object store) stays the source of truth. The CDN holds temporary copies.
• A user in Mumbai hits a Mumbai edge, a user in São Paulo hits a São Paulo edge, and both feel fast even though your origin sits in one data center in Oregon.

Two things make this work, and both matter for the interview:

• Geography. A round trip from Sydney to Virginia is roughly 200ms of pure network latency before you do anything useful. Sydney to a Sydney edge is single-digit milliseconds, call it 5ms. For a page that loads 40 assets, that per-asset gap compounds into the difference between snappy and sluggish, even with HTTP/2 multiplexing, because TLS setup and connection warm-up still pay the distance tax.
• Offload. Every request the edge serves from its cache is a request your origin never sees. A good config absorbs 90 to 99 percent of traffic for static content. At 99 percent offload your origin sees 1 request for every 100 the edge handles, which is the difference between running 4 origin servers and running 400.

2How it works#

Follow one file through the system and the mechanics fall out.

• A user asks for logo.png.
• DNS resolves your CDN hostname to the IP of a nearby edge (usually via anycast or geo-aware DNS).
• The edge checks its local cache.
• Fresh copy present: that is a cache hit and it responds immediately.
• Nothing fresh: that is a cache miss, so it fetches from origin, stores a copy, then responds. The next user in that region gets a hit.

The whole game is maximizing hits and keeping cached copies correct. Four mechanics control that:

• TTL (time to live). Your origin sends headers like Cache-Control: max-age=86400, telling the edge "good for a day" (86400 seconds). After it expires the edge revalidates.
• Cache key. The edge decides what counts as "the same request." By default it is the URL, but you can include or exclude query strings, headers, and cookies. Get this wrong and you either cache too little (every variation is a miss) or serve the wrong thing to someone.
• Revalidation. With ETag or Last-Modified the edge asks origin "has this changed?" and origin can reply 304 Not Modified with no body. A cheap way to confirm freshness without re-downloading a 2MB image to learn it is unchanged.
• Invalidation / purge. When you ship new content you tell the CDN to drop the old copy. Section 5 covers why this is painful.

One instinct to kill: "it's dynamic, so no CDN." That is wrong.

• Modern CDNs do dynamic acceleration for content that cannot be cached at all.
• Routing a request over the CDN's optimized backbone, with warm TLS connections held open between edge and origin, beats the open internet.
• You skip the TLS handshake cost on every hop and ride better peering.

Here is the same hit-and-miss logic as a time-ordered view:

3A worked example: image serving for a 50M-user app#

Put real numbers on it, because this is the back-of-envelope an interviewer will push you toward.

The setup.

• Product images for an e-commerce app.
• 50 million daily active users, each session loading about 40 images.
• Average image size 200KB after compression.
• Origin is one region, us-west-2.

Request volume.

• 50M users times 40 images is 2 billion image requests a day.
• That is roughly 23,000 requests per second on average.
• Peak around 70,000 RPS (a 3x peak-to-average factor is a safe number to say out loud).

Bytes.

• 2 billion requests times 200KB is 400TB served per day.

Now the CDN math at a 97 percent hit rate, which is realistic for versioned image assets:

• Origin requests: only 3 percent miss, so 70,000 peak RPS times 0.03 is about 2,100 RPS hitting origin. One modest fleet handles that. Without the CDN, origin eats all 70,000 RPS, a 33x difference in fleet size.
• Origin bandwidth: 400TB times 0.03 is 12TB a day from origin, versus 400TB. The other 388TB serves from edges near users.
• Latency: a hit from a nearby PoP returns in roughly 10 to 30ms. A miss pays the user-to-edge hop plus the edge-to-origin round trip, so a Sydney user on a miss might see 220ms. At a 97 percent hit rate, p50 and p90 sit in the hit range and the miss cost only shows up in the p99 tail. That tail is exactly why you care about hit rate, not just average.

Cost sketch.

• CDN egress runs on the order of 2 to 8 cents per GB depending on region and commit.
• 388TB a day at 4 cents per GB is about 15,500 dollars a day, call it 465,000 dollars a month.
• That is a real line item, and it is the number that justifies negotiating a committed-use discount.
• Serving the same bytes straight from origin egress is usually more expensive per GB and forces a far larger origin fleet, so the CDN wins on both axes.

The point of the numbers is to show both wins are measurable: a 33x smaller origin fleet and a 25x bandwidth offload, from one config choice.

4Trade-offs#

This is where interviews are won or lost. A CDN is not free magic. Here is what you actually trade.

Freshness versus hit rate.

• Long TTLs give high hit rates and low origin load, but users may see stale content after a change.
• Short TTLs keep content fresh but push traffic to origin and slow users on misses.
• The standard fix: content-hashed filenames (app.a9f3c1.js) with a one-year TTL, plus a short TTL on the HTML that references them.
• The HTML changes, points at new hashed files, and you never purge anything. State that pattern out loud.

Consistency.

• A CDN is an eventually consistent cache by nature.
• Right after a purge, different PoPs may serve different versions for a few seconds to minutes.
• For a marketing site, fine. For a config file that controls feature flags, dangerous.
• Know which of your content can tolerate it.

Cost shape.

• Pricing is mostly egress bandwidth plus request count.
• For high-volume static content it is cheaper than serving from origin because of offload.
• Egress at scale is a real line item (see the 465,000 dollars a month above).
• Some providers charge extra for cache invalidations or for traffic in expensive regions like India or South America.

Here is how I weigh a few common choices:

Push versus pull.

The thundering herd.

• On a popular cache miss, many edges or many users can hit origin at once.
• Good CDNs collapse these into a single origin request (request coalescing).
• Concrete case: a premiere drops and 500,000 viewers request segment 1 in the same 2 seconds against a cold edge.
• Without coalescing the edge opens 500,000 origin fetches and origin falls over.
• With it the edge sends 1 fetch and fans the one response out to all 500,000.
• If you are designing a CDN yourself, you need this.

5Where it is used#

Reach for a CDN when any of these are true.

• Static assets at scale. Images, CSS, JS, fonts. The bread and butter. A news site or e-commerce catalog serves the bulk of its bytes this way, as the worked example showed.
• Video and large downloads. Streaming pushes nearly all its traffic through CDNs, often using segmented formats (HLS/DASH) where each 2-to-10-second chunk is a separate cacheable file. Game patches and OS updates work the same way: one 50GB file, millions of identical requests, perfect for edge caching.
• Global APIs with cacheable responses. A product listing that changes a few times a day. Cache the GET at the edge with a 30-second TTL and you cut both latency and database load.
• Absorbing attacks. The edge sits in front of origin, so CDNs double as a first line of defense against traffic floods and act as the home for a web application firewall. The edge soaks up junk before it reaches you.
• TLS termination at the edge. Ending the encrypted connection near the user and keeping a warm link to origin shaves real milliseconds off every request, since the handshake happens over the 5ms edge hop, not the 200ms origin hop.

In a typical web system the CDN sits between the user and your load balancer. Static requests resolve entirely at the edge. Dynamic requests pass through to your origin.

6Common pitfalls and gotchas#

I have debugged every one of these in production. They show up in interviews too.

Caching things you should not.

• The classic disaster: an edge caches a page carrying a Set-Cookie with a session token, then serves that cookie to the next user. Now everyone is logged in as one person.
• Anything personalized must either not be cached or must vary the cache key correctly.
• When in doubt, mark user-specific responses Cache-Control: private, no-store.

Forgetting the cache key dimensions.

• API returns different content by Accept-Language but the cache key ignores it: the first visitor's language gets served to everyone.
• Key on something high-cardinality like a full cookie and your hit rate collapses toward zero, so the CDN becomes a slow pass-through that is worse than no CDN.

Treating invalidation as instant.

• Purges take time to propagate across hundreds of PoPs, typically seconds to a few minutes, and some are eventually consistent.
• Assume "I purged, so the old file is gone everywhere now" and you will ship bugs.
• Design so you rarely need to purge. Versioned URLs make this a non-issue.

Cold cache after deploy or purge.

• Right after a global purge your hit rate drops to zero and every request stampedes origin.
• In the worked example that is a jump from 2,100 origin RPS to 70,000 in an instant, a 33x spike that can knock origin over.
• Stagger purges, or use versioned URLs so old and new coexist.

Ignoring the first-byte penalty on misses.

• A cache miss is slower than no CDN at all for that one request, because it adds the user-to-edge hop plus the edge-to-origin hop.
• Your p99 includes these cold misses.
• If someone says "a CDN made everything faster," ask them about the miss path.

Compression and range requests.

• Serving uncompressed assets, or not supporting byte-range requests for video seeking, quietly wastes bandwidth and breaks playback resume.
• Easy to forget, easy to check.

7How it shows up in interviews#

When I ask about CDNs I am probing whether you understand the two wins (latency and offload) and whether you know a cache has costs.

What I listen for:

• Do you distinguish static from dynamic, and know the CDN helps both for different reasons?
• Can you reason about hit rate? If I say "our hit rate is 60 percent, is that good?" a strong candidate asks what the content is, checks the cache key, and checks whether TTLs are too short.
• Do you handle invalidation sanely? "Versioned, content-hashed URLs with long TTL" is a green flag. "We purge on every deploy" invites follow-ups.
• Do you name failure modes? Thundering herd on a miss, caching private data, stale config.

That strong answer shows you understand the levers, the numbers, and the sharp edges. It is also honest: it says what is cached, what is not, and why.

Next, try the System Design Question walkthrough on designing a global photo or video sharing service, where CDN caching strategy is the center of the problem, not a footnote.