Design YouTube

1Clarifying the requirements#

Pin down what "YouTube" means for this hour before drawing a single box. The worst move a candidate can make is to start designing the recommendation feed when the interviewer cares about the upload-and-play path. So ask a few sharp questions and let the answers steer the design.

Candidate: Are we designing the whole product, or the core video flow? Upload, process, watch. Interviewer: Core flow. Assume search, comments, and recommendations exist but are not the focus. I want upload, transcode, and playback done well.

Candidate: Do we need live streaming, or only on-demand uploaded video? Interviewer: On-demand only. Live is a different beast, call it out but don't build it.

Candidate: What scale? Are we at launch, or are we already global? Interviewer: Global. Think a billion-ish users, hundreds of hours uploaded per minute.

Candidate: Last one. Is play latency more important than upload latency? Interviewer: Yes. People will wait for an upload to process. Nobody waits for a video to start.

What those answers just told me:

• The interview is about the upload-and-play path, not the recommendation feed.
• "Nobody waits for a video to start" is the hinge: optimize the read path hard, let the write path be slow and async.
• The one metric worth protecting in production is time-to-first-frame, not time-to-processed.
• Live is explicitly out, so I name it and move on instead of half-building it.

Functional requirements

• Upload a raw video file of arbitrary size and format.
• Transcode it into multiple resolutions and streaming formats.
• Stream video to viewers with fast startup and smooth playback.
• Store and serve metadata: title, description, owner, thumbnails, view count.
• Track views and basic engagement.

Non-functional requirements

• Playback start under about 500 ms at the p50, globally.
• Durable uploaded content. Losing someone's video is unacceptable, so I target eleven nines of object durability.
• High read availability. A watch page must load even if the upload pipeline is fully degraded. I want the read path at 99.95%+ independent of upload health.
• Upload processing can take minutes. A few minutes for the 4K rendition is fine.
• Cost-aware. Bandwidth is the dominant expense, so the design respects that from the first box.

2Back-of-the-envelope estimation#

Get the shape of the traffic in your head, because it decides the architecture. Use round numbers and say they are round: the point is the order of magnitude, not three significant figures.

Users and reads.

• 1B daily active users, each watching about 5 videos a day, so 5B views/day.
• Over 86,400 seconds that is about 58,000 view-starts/second average.
• Peaks run 3x to 5x, so design the read path for roughly 250,000 requests/second at the edge.
• Most of that never touches origin: it is served by the CDN.

Writes.

• Uploads are tiny by comparison. Say 500 hours of video uploaded per minute.
• That is about 8.3 hours per second of source material, maybe a few thousand upload requests/second.
• Low QPS, but each request kicks off a heavy processing job.
• Cheap reads from cache versus expensive writes: that asymmetry justifies throwing the whole read path behind a CDN and keeping the app tier thin.

Storage.

• 500 hours/min x 60 x 24 = 720,000 hours uploaded per day. At about 1 GB per source hour, that is 720 TB/day of raw upload.
• We also transcode into about 6 renditions (144p up to 4K). All qualities combined add roughly 2x to 3x the source.
• Call the stored footprint about 3 GB per source hour after transcoding.
• 720,000 hours/day x 3 GB = about 2.1 PB/day, roughly 770 PB/year. This is an object-storage problem, full stop. No database holds this.

Bandwidth (the number that bites).

• 5B views/day, average watch about 8 minutes, average bitrate about 2.5 Mbps.
• Egress per view is about 2.5 Mbps x 480 s / 8 = about 150 MB. Times 5B is about 750 PB/day served.
• If even 5% of that hit origin, origin would melt. The CDN is not an optimization, it is the product.

Compute and the write cost.

• Compute scales with uploads, which are small, so the transcode fleet is a few thousand QPS of jobs, not millions.
• Each job is heavy (a 2-hour 4K source is a lot of CPU), so the fleet autoscales on queue depth, not request count.

3Core APIs#

Keep these small and boring on purpose, with one twist: upload is a multi-step handshake, not one POST.

• The client never streams bytes through the app tier.
• We hand back a signed URL so raw video goes straight to object storage.
• Routing 720 TB/day of raw upload through the app servers would be absurd, so the Upload Service only ever brokers a signed URL.

POST /v1/videos
Authorization: Bearer <token>
Content-Type: application/json

{
  "title": "Backyard rocket test",
  "description": "it worked, mostly",
  "visibility": "unlisted",
  "sizeBytes": 2147483648
}

{
  "videoId": "vid_8f21ab",
  "uploadUrl": "https://uploads.cdn.example.com/raw/vid_8f21ab",
  "uploadMethod": "PUT",
  "status": "AWAITING_UPLOAD"
}

The client PUTs raw bytes to uploadUrl directly:

• The upload is chunked and resumable, so a dropped connection on a 50 GB file resumes instead of restarting.
• When storage confirms the object landed, it fires an event that starts the pipeline.
• The client polls status while renditions cook.

GET /v1/videos/vid_8f21ab/status

{
  "videoId": "vid_8f21ab",
  "status": "TRANSCODING",
  "readyRenditions": ["144p", "360p"],
  "pendingRenditions": ["720p", "1080p", "4k"]
}

Playback does not return a file. It returns a manifest that lists the quality ladders, and the player adapts.

GET /v1/videos/vid_8f21ab/manifest

{
  "videoId": "vid_8f21ab",
  "manifestUrl": "https://cdn.example.com/vid_8f21ab/master.m3u8",
  "thumbnails": "https://cdn.example.com/vid_8f21ab/thumbs/",
  "durationSec": 612
}

Two choices worth defending:

• The manifest endpoint returns a CDN URL, not bytes. The app tier hands back a pointer and gets out of the way. The player talks to the edge for every segment after that.
• Status is a poll, not a blocking call. Processing takes minutes, so the client watches renditions flip to ready rather than holding a connection open.

4High-level design#

The spine is two paths, so draw it fast:

• Write path: an async pipeline that turns raw bytes into streamable renditions.
• Read path: a thin metadata lookup plus a fat CDN.
• The client uploads to the Upload Service only to get a signed URL, then pushes bytes to object storage directly.
• Storage emits an event onto a queue, transcode workers pick it up, chunk and transcode in parallel, write output back to storage, and mark the video ready.

Write path.

• You request an upload URL and PUT raw bytes straight to object storage.
• Storage emits an "upload complete" event onto the Transcode Queue.
• A fleet of workers picks up the job, splits the video into chunks, and transcodes each chunk into every rendition in parallel.
• Workers write output back to storage and mark the video ready in the Metadata DB.

Read path.

• You hit the API and get a manifest (a CDN URL, not bytes).
• You stream segments from the nearest CDN edge.
• The edge pulls from origin storage on a miss, then caches.
• Metadata lives in its own DB with a cache in front for the hot stuff like view counts and titles.

Why split it this way.

• The two paths have opposite shapes: writes are low QPS and heavy, reads are enormous and cacheable.
• The queue decouples upload from processing, so a backlog just means slower processing, never a dropped upload or a stalled API.
• The CDN sits on the hot path for bytes while the metadata DB only ever serves small JSON, which is exactly why it stays small and fast.

5Data model#

Use a few stores, each picked for its access pattern. Metadata is small and structured; the heavy data lives in object storage and is referenced by key, never stored inline. Never put a byte of video in a row.

videos (relational or well-indexed document store, written once at upload and read millions of times):

• The videos row is written once and read millions of times, so it caches beautifully.

renditions (one row per quality-and-format, so the player can build its ladder):

• The ready flag per rendition is what lets us flip a video to playable on 144p while 4K is still cooking.

view_counts (kept separate, written at a completely different rate):

• Split out because the counter's write pattern is nothing like the mostly-static videos row.
• Jamming a hot counter into the main table creates contention on a row everyone also reads.
• The counter is written hundreds of thousands of times a second on a viral video, so it needs its own home, covered in 6.3.

6Deep dives#

6.1 The transcoding pipeline#

This is the signature hard part, so it is where I spend the most time.

• A single 2-hour 4K upload can be 50 GB.
• One worker transcoding it start to finish stalls that video behind one machine for an hour, and you cannot parallelize.
• So we chunk it.

How a worker handles a job.

• Quick probe for codec, duration, and resolution.
• Split the source into independent segments of a few seconds each, along keyframe boundaries.
• Drop a transcode task per chunk per rendition onto the queue, so hundreds of workers chew through one video at once.
• A separate step stitches per-chunk outputs into each rendition's manifest.
• The moment the lowest rendition finishes, mark the video playable so viewers start watching while higher qualities cook.

This "lowest rendition first" rule is what turns a 20-minute transcode into a few-seconds time-to-watchable.

Three things I would defend here:

• Chunk on keyframe boundaries. Otherwise segments cannot decode independently and you get artifacts at the seams.
• Every stage is idempotent and keyed by chunk ID. Workers die mid-job constantly at this scale. A redelivered message must re-produce the same output object, so writes are deterministic on the chunk key: output lands at a path derived from videoId/rendition/chunkId, and a retry overwrites the exact same object.
• The fan-out is the whole point. One 2-hour source becomes thousands of independent tasks (chunks x renditions), which is why the worker fleet autoscales on queue depth rather than on upload count.

6.2 Delivery and the CDN#

The read path lives or dies on cache hit rate, and since egress is the number that drives the system, this is the second deep dive I always force.

How delivery works.

• We serve over HLS or DASH: the source is cut into short segments (say 4 to 6 seconds each).
• The player downloads a master manifest, picks a quality, and pulls segments one at a time.
• This is what makes adaptive bitrate work: when the viewer's bandwidth drops, the player quietly switches to a lower rendition on the next segment. No rebuffer if we do it right.

Why segments cache forever.

• Segments are immutable and named by content, so the edge can hold them with an effectively infinite TTL.
• A popular video gets pulled to an edge once and then served millions of times without touching origin.
• Immutable plus content-addressed means no invalidation to worry about.

Why the mid-tier matters, in numbers.

• The tiered cache is edge, then a regional mid-tier, then origin.
• The mid-tier shields origin so a cold video pulls from origin once per region instead of once per edge.
• If the hit rate is 95%, origin sees 5% of 750 PB, about 37 PB/day. Push it to 99% and origin sees 7.5 PB/day.
• That one percentage point is tens of petabytes of egress, which is why I obsess over it.
• For the long tail of rarely-watched videos, accept more origin reads but keep those renditions in cheaper, colder storage.

6.3 View counts without melting a row#

A viral video gets hundreds of thousands of view events per second.

• You cannot do UPDATE videos SET count = count + 1 for that: you serialize every writer on one row.
• So we never write the counter synchronously.

The aggregation path.

• View events go onto a stream.
• A set of aggregators keep in-memory per-video tallies and flush batched increments every few seconds.
• The displayed count is read from cache.
• It is eventually consistent, and I say that out loud: a viewer seeing 1,000,204 instead of 1,000,250 for a few seconds is completely fine.

Why it is worth it.

• A flush every 5 seconds turns a million raw increments into one batched update. The write amplification savings are huge.
• For anti-fraud we dedupe within a window so a single session replaying does not inflate counts, but that is a filter on the stream, not something the hot path cares about.

6.4 Resumable upload and the direct-to-storage handshake#

The fourth piece I like to defend is why the client PUTs to storage and never to the app tier.

• A 50 GB upload over a flaky mobile connection will drop.
• If every drop restarts the whole file, uploads from real-world networks basically never finish.

So the signed URL points at resumable storage.

• Object storage supports chunked, resumable multipart upload: the client uploads in parts, each part is acknowledged, and a dropped connection resumes from the last acked part.
• The app tier only ever issues the signed URL and later consumes the "upload complete" event.
• This keeps 720 TB/day of raw bytes entirely off our servers and makes the write path survivable on bad networks.

7Bottlenecks and trade-offs#

Bandwidth cost is the pressure point.

• At 750 PB/day of egress, the design has to maximize cache hit rate and push bytes from the cheapest possible location.
• The trade-off: aggressive caching and multiple renditions cost a lot of storage, but storage is cheap relative to repeated origin egress.
• I would happily store 3x the data to avoid serving it from origin twice.

Transcode backlog.

• If uploads spike (a major event), the queue grows and processing lags.
• Prioritize the lowest rendition first so videos become watchable fast, and autoscale the worker fleet on queue depth.
• Worst case, high-res renditions lag by minutes, which our requirements allow.
• What is not acceptable is dropping an upload, which the durable queue plus idempotent workers prevent.

The hot-video thundering herd.

• When a brand-new video goes viral, every edge misses at once and stampedes origin.
• The regional mid-tier plus request coalescing (collapse concurrent misses for the same object into one origin fetch) handles this.
• Pre-warming top creators' uploads to edges helps too.

Metadata consistency.

• Eventual consistency for counts, and read-your-writes only where it matters: the uploader seeing their own video's status.
• A viewer seeing a stale title for a second is fine.
• I would not pay the latency and availability cost of strong consistency across a global metadata store for data nobody needs to be exact.

Storage tiering.

• The access pattern is brutally skewed: a small fraction of videos get most of the views, and the long tail is watched almost never.
• Keep hot content on fast storage near edges and demote cold content to cheap archival tiers, accepting slower first-byte on rarely-watched videos.
• Optimizing the tail for speed would waste money on footage nobody requests.

Single points of failure.

• The metadata DB is the one thing that, if it is down, takes playback with it.
• So it gets read replicas per region, a cache that can serve stale on failure, and a playback service that degrades gracefully: with a cached manifest it still serves the video.
• Bytes keep flowing from the CDN even during a metadata incident. This is how we hold the read path at 99.95%+ regardless of what the write path is doing.

Observability and SLOs.

• SLOs as measured targets: playback start p50 under 500 ms, read-path availability at 99.95%+, and time-to-watchable (upload to lowest rendition ready) in the low tens of seconds.
• Watch: CDN hit rate per tier (edge vs regional), origin egress, transcode queue depth and lag, failed-job rate, p50 playback start.
• When a number moves, I already know which lever it maps to.

Always let egress, not compute, pick where the architecture spends its effort.

8Summary#

The decisions that matter:

• Two isolated paths. A slow async chunked transcode pipeline on the write side, a thin metadata lookup plus a fat CDN on the read side, glued only at the metadata DB. Keeping them separate is the single most important call.
• CDN as the core. Immutable content-addressed segments cache forever, a tiered edge-then-regional-then-origin cache keeps origin egress tiny, and one point of hit rate is worth tens of petabytes.
• Chunked transcode, lowest rendition first. One source becomes thousands of idempotent chunk jobs so the fleet parallelizes, and the video flips to playable on 144p in seconds.
• Upload direct to storage. Signed URL plus resumable multipart keeps 720 TB/day off the app tier and survives flaky networks.
• Hot counter on its own path. View events stream into a batched aggregator so a viral video never contends with the mostly-static video row.
• Out of scope by choice: live streaming, search, comments, recommendations. Naming them keeps me honest about what 45 minutes covered.

For how specific companies probe this one, see our related guides on designing a CDN, a distributed message queue, and a scalable object store.