Design WhatsApp / Messenger

1Clarifying the requirements#

Pin down what "chat" means before drawing a single box. The scope swings wildly between a toy and a billion-user firehose, so treat these answers as load-bearing. Here is how I would open:

Candidate: Are we doing one-to-one only, or group chats too? And do we need media, or just text?

Interviewer: Both one-to-one and groups. Groups up to, say, 500 members. Text plus images and short video.

Candidate: Do we need delivery and read receipts? The double-tick behavior?

Interviewer: Yes, sent, delivered, and read. Those matter a lot to users.

Candidate: Last question on scope: end-to-end encryption, and does message history live on the server forever?

Interviewer: Assume E2E encryption is a goal but keep it light. And no, the server should not keep messages after they are delivered to all recipients. Treat the server as a relay with short-lived storage.

What those answers just told me:

• The interview is about real-time delivery and fan-out, not storing chat.
• "Relay with short-lived storage" is the whole personality of the design: we are not Slack, the server is a mailbox, not an archive.
• Delete-on-delivery changes the storage math (gigabytes, not petabytes) and the consistency story (we never promise a searchable global history).
• I want the interviewer to see I caught the implication, not just the answer.

Functional requirements

• Send and receive one-to-one messages in near real time.
• Group chat up to 500 members.
• Delivery states: sent, delivered, read.
• Media messages (images, video) up to a size cap.
• Offline delivery: if you are offline, you get messages when you reconnect.
• Presence: online / last seen.
• Multi-device: the same account on phone plus desktop.

Non-functional requirements

• Scale to 1B daily active users, tens of billions of messages a day.
• Latency: p99 end-to-end delivery under 1 second for online users.
• Availability 99.99%. People notice chat being down instantly.
• Durability until delivered: a message the server accepted must not vanish before it reaches every recipient.
• Ordering: messages in a conversation arrive in the order the sender sent them.

2Back-of-the-envelope estimation#

Get the scale honest before committing to an architecture, because here one number decides everything. I will flag it when I reach it.

Users and messages.

• 1B DAU, each sending about 40 messages a day, so 40B messages a day.

40B / 86,400 sec ~= 463K messages/sec written (average)

• Chat is spiky. Evening peaks run 3x to 4x average.
• Design for roughly 1.5M messages/sec at peak writes.

Reads vs writes (this is the one that matters).

• A one-to-one message is read by one person. A 500-member group message is read by up to 499.
• If about 10% of messages are group with ~50 real recipients each, the fan-out multiplier is:

reads = 0.9 * 1 + 0.1 * 50 = 0.9 + 5 = ~5.9 deliveries per message sent

• Peak delivery rate lands around 9M messages/sec. Delivery, not acceptance, is 6x the work.
• This tells me the expensive machinery is fan-out and push, and the cheap part is accepting the write.

Connections.

• A big chunk of users are connected at once. Say 300M devices hold a live socket at peak.
• One box comfortably holds ~500K idle WebSocket connections.

300M / 500K = 600 gateway boxes (just for live sockets)

Storage (the number that justifies the stance).

• Messages are short-lived (deleted after delivery), but undelivered ones queue up.
• Assume 2% of messages are in flight for an offline recipient. Average payload with metadata ~300 bytes.

40B/day * 300 bytes = 12 TB/day flowing through
undelivered buffer at 2% held ~3 days = 40B * 0.02 * 300B * 3 ~= 720 GB

• 720 GB is tiny, precisely because we delete on delivery. That number is the whole argument for "relay, not archive."
• Keep history instead and you write 12 TB a day into cold storage forever: a completely different and more expensive product.

Media.

• Say 5% of messages carry media averaging 300 KB.

40B * 0.05 * 300KB = 600 TB/day of media uploads

• Media lives in object storage behind a CDN. It never touches the message pipeline except as a URL.
• The "message" I move at 9M/sec is a 300-byte envelope with a pointer, not a photo.

3Core APIs#

Clients do not HTTP-poll for messages. The read path is a persistent connection. Setup, media, and group management stay plain requests.

POST /v1/messages
Authorization: Bearer <token>
Content-Type: application/json

{
  "client_msg_id": "c0ffee-8821",
  "conversation_id": "conv_9f3",
  "type": "text",
  "ciphertext": "base64...",
  "media_url": null
}

200 OK
{ "server_msg_id": "01H...", "ts": 1730600000123, "state": "sent" }

Two design choices worth defending:

• client_msg_id is the idempotency key, and it does real work. A flaky-mobile retry dedupes on it and returns the same server_msg_id, so a spotty subway ride never produces duplicate bubbles.
• The POST returns the moment the message is durably in the inbox, before it reaches the recipient.

POST /v1/media/upload-url
{ "mime": "image/jpeg", "size": 240000 }

200 OK
{ "upload_url": "https://blob...", "media_url": "https://cdn..." }

POST /v1/receipts
{ "server_msg_id": "01H...", "state": "read" }

GET  /v1/presence?user_ids=u1,u2,u3

The media flow is a two-step:

• Client asks for a signed upload URL.
• Client uploads the bytes straight to blob storage, never through our servers.
• Client sends a normal text-type message whose media_url points at the CDN.
• That keeps 600 TB/day of photos off the hot path.

Over the live socket, the server pushes frames like:

{ "op": "deliver", "server_msg_id": "01H...", "conversation_id": "conv_9f3", "from": "u7", "ciphertext": "..." }
{ "op": "receipt", "server_msg_id": "01H...", "state": "delivered" }

4High-level design#

The spine is standard, so draw it fast:

• A phone opens one long-lived WebSocket to a Connection Gateway.
• The gateway owns sockets and knows which user sits on which box. Nothing else.
• A Message Service assigns an ID and sequence, drops the message in the recipient's inbox, and asks a Router where that recipient is connected.
• The router pushes the frame to the right gateway, which writes it down the socket.

The key decisions baked into this picture.

• The gateway is dumb and stateless about content. It holds sockets and forwards. Scale by adding boxes for more of the 300M sockets, and lose one without losing a single message, because the message was never the gateway's to keep.
• The routing table in Redis maps user_id -> gateway_id (and device). It is the phone book: the gateway writes the mapping on connect, clears it on disconnect. Every one of those 9M deliveries/sec does at least one lookup here, so it gets its own deep dive.
• The inbox queue is per-user durable storage of undelivered messages. Offline recipient? The message sits here until they come back and drain it. This is the mailbox.
• The Push Notifier wakes sleeping phones through Apple's APNs and Google's FCM so the app reconnects and pulls. We cannot hold a live socket to a phone the OS put to sleep, so this is how offline delivery actually gets kicked off.

The flow in one line.

• Accept fast, persist to the recipient's inbox, then try to deliver live.
• Live delivery fails because the recipient is offline? The message is already safe in the inbox, so we fire a mobile push.
• I never block the sender on the recipient being reachable.

5Data model#

Use a few stores, each shaped around exactly one query, so a single partition read answers it. Writes are relentless at 1.5M/sec, so the hot stuff goes in a wide-column store (Cassandra-style). No joins, ever.

Messages (short-lived, per conversation):

• Partitioning by conversation_id makes "give me this conversation in order" a single-partition range scan on seq.

Inbox / per-recipient delivery queue:

• Keyed per device, not per user, because "delivered" means every one of your devices pulled it. This row is the unit the drain-and-reap logic works on.

Conversations:

Presence (Redis, not the DB):

• Presence changes thousands of times per user per day and nobody needs it durable, so a persistent write per heartbeat would be waste.
• A short TTL does the cleanup: a missed heartbeat lets the key expire and the user flips to offline on its own. No janitor process sweeping a billion rows.

6Deep dives#

6.1 Delivery, receipts, and offline#

This is the core loop and the signature hard part. Walk the full path for one message, online and offline.

Why the ack lands where it does.

• The sender gets "sent" the moment the message is durably in the inbox, not when the recipient sees it.
• That gives a fast single tick and decouples the sender's latency from whether the recipient's phone is even on.
• The delivered receipt (second tick) waits for Gateway B to confirm the socket write and the phone to ack.
• The read receipt is a separate client-initiated event, fired when the chat is actually on screen, which is why read can lag delivered by minutes.

When the recipient is offline.

• The routing lookup returns nothing, so we stop and the message stays queued in the inbox.
• The Push Notifier sends a quiet push to APNs or FCM.
• The phone reconnects, the gateway tells the Message Service "user B device D is back," and we drain their inbox: deliver everything with state = queued in server_msg_id order, then flip rows to delivered.
• Because server_msg_id is a ULID, that drain is already in roughly the right time order for free.

At-least-once means dedupe on the client too.

• We may push, lose the socket before the ack, and redeliver on reconnect.
• Client dedupes on server_msg_id, send side dedupes on client_msg_id.
• Both ends idempotent, so retries are safe and never create a duplicate bubble.

6.2 Group fan-out#

A 500-member group is where naive designs fall over, and it is the direct consequence of that 9M/sec number. The question is where fan-out happens.

• Write fan-out: on post, the Message Service writes one copy into each of the 499 recipients' inboxes immediately. Delivery is then a normal per-user drain. Reads cheap, writes heavy.
• Read fan-out: store the message once per conversation and let each member pull it, tracking per-member read cursors. Writes cheap, reads do the work.

For chat, delivery latency wins almost everywhere, so write fan-out is my default:

• 499 inbox writes at 300 bytes each is roughly 150 KB of writes, which is nothing.
• But the math inverts at the extreme: write fan-out to a million inboxes on every message would melt the cluster.
• So the rule I draw on the board: groups under ~1,000 members get write fan-out, above that read fan-out with a shared cursor.

Fan-out runs async through a queue.

• The sender's ack does not wait on 499 writes.
• The sender gets "sent" as soon as the message is persisted to the conversation log and enqueued for fan-out.
• Members get it a few tens of milliseconds later.
• Same "accept fast, deliver async" move from the one-to-one path, just multiplied, which is why the sender never feels the size of the group.

6.3 Ordering#

Users will scream if message B shows above message A when A was sent first. I do not rely on wall-clock timestamps across machines, because clocks drift and a sender and recipient on different continents will disagree on "when." Ordering is per conversation, the only scope a user ever perceives.

How the sequence works.

• The Sequencer owns a monotonic counter per conversation_id. Every accepted message gets the next seq.
• A single conversation's writes route to one sequencer shard (hash on conversation_id), so the counter is strictly increasing with no cross-machine coordination.
• That gives a total order within a conversation without a distributed lock.
• Clients render by seq, not by received time.

Out-of-order arrival is normal and handled.

• A client receiving seq 7 before seq 5 buffers 7 and waits for 5, or asks the server for the gap.
• The user never sees the reorder.

Two IDs doing two jobs, deliberately kept separate.

• seq is the authoritative in-conversation order: it is for correctness of ordering.
• ULID server_msg_id is for global uniqueness, dedupe, and storage locality.
• Mixing those two jobs into one field is a common mistake. Say the split out loud.

6.4 Connection management and presence#

Each gateway holds hundreds of thousands of sockets, up to that 500K figure. Presence is just "do I have a live socket for this user, and is the heartbeat fresh."

Heartbeats drive presence, TTL does the cleanup.

• The phone sends a ping every ~20 seconds.
• The gateway refreshes the Redis presence key with a 40-second TTL.
• Miss two pings and the key expires: the user is offline automatically, with no janitor sweeping a billion rows.

Do not broadcast presence to the whole graph.

• Showing "online" to a friend means subscribing to presence changes, and this is where people accidentally design an N-squared firehose.
• A user with 2,000 contacts going online is not 2,000 push events.
• Presence is pulled lazily: open a chat, the client asks for that one person's presence and subscribes just to them.
• We push presence updates only to the small set of users actively looking at that contact's chat right now.
• That keeps presence traffic proportional to open chat windows (a handful per user), not to the social graph (thousands per user).

7Bottlenecks and trade-offs#

The gateway fleet is the scary part.

• 600+ boxes, each holding 500K sockets. One box dies and 500K phones reconnect at once, tending to hit the same load balancer in the same second.
• That thundering herd can cascade: the reconnect spike knocks over the next box, which dumps another 500K.
• Defense: jittered reconnect backoff on the client (reconnect after a random 1 to 30 seconds, not instantly).
• Defense: spread gateways across many LB targets so the reconnect wave diffuses instead of landing on one door.

The routing table is a hot dependency.

• Every message does at least one Redis lookup to find the recipient's gateway. At 9M deliveries/sec that is at least 9M reads/sec hitting Redis.
• Shard Redis by user_id and cache the mapping on the Message Service for a few seconds.
• A just-moved user might get one failed push and a retry, which is far cheaper than keeping Redis perfectly fresh at 9M/sec.
• This is deliberately loose consistency chosen on purpose, and I want the interviewer to see the choice.

Hot conversations.

• A celebrity group or viral broadcast concentrates writes on one conversation_id shard, which is also where the sequencer counter lives, so it is a hot key on write and ordering at once.
• Mitigation: detect high-rate conversations and switch them to read fan-out (as in 6.2).
• Mitigation: relax strict sequencing for broadcast-style channels where exact ordering matters less than throughput.
• For a one-way announcement channel, nobody cares if two posts a millisecond apart swap, so spend that strictness budget elsewhere.

Delete-on-delivery vs multi-device.

• The requirement says delete after delivery, but multi-device breaks the simple version: a message is "delivered" only when all of a user's devices have pulled it.
• So the inbox is keyed per device, and a row is reaped only when every active device acks.
• A user with a dead second phone they never open would pin messages in the mailbox forever, quietly breaking the 720 GB estimate.
• So I add a hard TTL (say 30 days) regardless. That is a deliberate trade: lose an undelivered message to a long-dead device rather than grow the mailbox without bound.
• I say that trade-off out loud, because pretending the tension does not exist is worse than naming it.

E2E encryption limits the server.

• ciphertext is opaque, so the server cannot do server-side search, content spam filtering, or smart grouping. It routes bytes.
• Search and rendering move to the client, which holds the keys.
• Spam defense moves to metadata signals (send rate, graph patterns) instead of content, which is weaker.
• That is a real cost of the privacy stance, and I flag it as a known gap rather than claim we solved it.

Consistency model.

• This is an AP system in CAP terms. During a partition I keep accepting and delivering, and I tolerate temporary disagreement (you might see "sent" while the other side already read it, for a second).
• I will not block a send waiting for global agreement, because a chat app that stalls on partition is a dead chat app.
• The one firm guarantee: once the server acks a message, it will eventually reach every recipient device or expire by TTL, and within a conversation order is preserved.
• Everything else I am willing to let be eventually consistent.

This problem is not about storing chat, it is about pushing it. The 9M deliveries/sec number is the tell.

8Summary#

The decisions that matter:

• Persistent WebSocket per device through a dumb, stateless Connection Gateway. Scale by adding boxes, survive failures because content lives elsewhere.
• Server as a mailbox, not an archive. Per-recipient inbox rows, deliver, then delete. Keeps storage near 720 GB despite tens of billions of messages a day.
• Accept fast, deliver async. Ack the sender when the message is durably queued, not when it is read. Fan-out and delivery happen behind the ack.
• Write fan-out for normal groups, read fan-out above a ~1,000-member cap. Optimize delivery latency where it matters, protect against broadcast blowups where it does not.
• Per-conversation sequence numbers for ordering, not wall clocks. ULIDs for global uniqueness and dedupe, kept as a separate job from seq.
• At-least-once with idempotency keys on both ends (client_msg_id and server_msg_id) so retries are safe and never double-render.
• Presence in Redis with TTL heartbeats, pulled lazily per open chat rather than broadcast to the graph.
• AP under partition. Availability and low latency over strict global consistency, with the one firm promise that an acked message eventually reaches every device or expires.

For how specific companies probe this one, see our related guides on the engineering interviews at major social and infra companies.