Design Slack

1Clarifying the requirements#

Pin down what you are on the hook for before drawing a single box. A loose "design Slack" can mean ten different systems, and the scope call here changes the architecture. Here is how I would open:

Candidate: Are we building the full thing, workspaces plus channels plus DMs plus threads plus search, or just the real-time message core? Interviewer: Assume the core messaging product. Channels, DMs, threads, presence, and search. Skip video calls and the app platform.

Candidate: What is the latency target for a message showing up on another person's screen? Interviewer: Feels instant. Call it under 500ms at p99 for people in the same channel.

Candidate: Biggest channel size? That changes my fan-out call a lot. Interviewer: Most channels are small, under 50 people. But some announcement channels have 100,000 members. Design for both.

Candidate: Do we need message history to be durable and searchable forever? Interviewer: Yes. Messages are the product. Nothing gets silently dropped, and search over years of history has to work.

What those answers just told me:

• The interview is about fan-out and the socket tier, not CRUD.
• "Design for both" sizes means one fan-out policy will not cover a 10-person DM and a 100K channel, so I am building two paths from the start.
• "Messages are the product, nothing gets silently dropped" is the interviewer telling me durability is non-negotiable: I persist before I deliver, never the reverse.

Functional requirements

• Send and receive messages in channels, DMs, and threads.
• Real-time delivery to all online members of a channel.
• Presence: who is online, plus typing indicators.
• Full-text search across every message a user can see.
• Unread counts and read state, per channel per user.
• File uploads attached to messages.

Non-functional requirements

• Low latency delivery. Under 500ms p99 for same-channel peers.
• Durability. A sent message is never lost once acked.
• High availability on read and send. A deploy of the socket tier must not drop messages.
• Ordering. Messages in one channel appear in a consistent order for everyone.
• Scale to tens of millions of daily users, giant channels included.

2Back-of-the-envelope estimation#

Get the scale in your head, because it decides the topology. One number drives everything, and I will flag it when I reach it.

Users and connections.

• 30 million daily active users.
• Knowledge workers leave the app open all day, so this is not a spiky social app. Say 40% online at peak.
• That is about 12 million live WebSocket connections held at once.

Writes.

• An active user sends maybe 40 messages a workday. 30M x 40 = 1.2B messages/day.
• Spread over an 8-hour workday that is roughly 42,000 writes/second average.
• Call it 120,000 writes/second at peak. That rate is honestly modest. A single well-sharded wide-column cluster eats it.

Read amplification (this is the one that matters).

• Each message fans out to every online member of the channel.
• Average channel has maybe 15 online members, so 42K writes become about 630K delivery events/second average.
• At peak with big channels it spikes to several million delivery events/second.
• Reads beat writes by at least 15 to 1, and that ratio explodes on an announcement channel where one write implies 100,000 deliveries.

Storage.

• Average message is about 300 bytes plus metadata, call it 1 KB with indexes.
• 1.2B/day x 1 KB = 1.2 TB/day, about 440 TB/year before replication.
• Files are separate and larger, so they go to object storage and a CDN, never into the message rows.

Search index.

• Text is maybe 100 bytes per message.
• The inverted index runs 1.5x to 3x the raw text, so budget a few hundred TB of index spread across shards.

3Core APIs#

Keep these small and boring on purpose. The send path is HTTP for durability, delivery is over the WebSocket, and keeping them separate is the seam the whole design pivots on.

POST /api/channels/{channelId}/messages
Authorization: Bearer <token>
Content-Type: application/json

{
  "clientMsgId": "a1b2-c3d4",
  "text": "shipping the fix now",
  "threadId": null
}

200 OK
{ "messageId": "01Hinclude", "ts": "1730600000.004500", "channelId": "C123" }

Two design choices worth defending:

• clientMsgId is the idempotency key, and it is doing real work. A client retrying after a flaky network dedupes on it and gets the same messageId back instead of double-posting. Without it, every mobile user on a subway sends duplicates.
• The POST returns the moment the message is durably stored and published, before a single delivery happens.

Reading history is a backward range scan from a cursor, not offset pagination:

• Offset pagination on an infinite scrollback is a trap: new messages at the head shift everything, giving duplicates or gaps.
• The cursor ts maps directly onto how the message store is laid out (one channel's history in one partition, sorted by time), so the server resumes a range scan without re-counting from zero.

GET /api/channels/{channelId}/history?before=<ts>&limit=50
GET /api/search?q=deploy+rollback&in=C123&from=user%40co

Search carries the channel and sender filters into the index so the query is scoped, not a full-corpus scan. The real-time channel, over WebSocket:

{ "type": "message", "channel": "C123", "msg": { "id": "...", "text": "..." } }
{ "type": "presence", "user": "U9", "status": "active" }
{ "type": "typing", "channel": "C123", "user": "U9" }

The client also sends up the reverse direction: on connect it ships a cursor per channel (last ts seen) so the server knows exactly what to replay. That one detail is what makes gateway crashes survivable, and I will come back to it.

4High-level design#

The spine has four tiers, so draw it fast:

• A stateless API tier for sends and history.
• A stateful gateway tier holding the WebSocket connections.
• A pub/sub bus that routes messages to the right gateways.
• The storage layer underneath.

Write path.

• You POST a message.
• The API service writes it durably to the Message DB (the source of truth) and assigns a ts.
• It publishes one fan-out event, then the HTTP response returns, so sending feels instant.
• Fan-out and delivery happen asynchronously behind the bus.

Read path.

• History is plain HTTP against the message store, cache-fronted (recent messages per channel sit in Redis).
• Live delivery is a push: gateways subscribe to the channels their connected users care about and push the message down the matching sockets.
• Search is its own subsystem with its own index, fed asynchronously from the write path.

Why split it this way.

• The send path and the delivery path have opposite shapes: sends are small and must be durable, delivery is enormous and tolerates a bus.
• The sender only waits on one durable write plus one publish, not on 100,000 deliveries.
• A fan-out backlog can never slow down or drop a send, and indexing lag never slows delivery.

Monotonic, time-ordered message IDs.

• I want the ts within a channel monotonic and time-ordered, so I can range-scan and paginate without a separate sort key, and so clients sort by ts instead of arrival time.
• That doubles as the message id inside the partition, and it is the thing that makes per-channel ordering correct even when sockets deliver out of order.

5Data model#

Use a few stores, each picked for its access pattern. Messages and channel membership are different beasts, so I do not force them into one database.

Messages (wide-column store like Cassandra, partition by channel_id, clustered by ts desc):

• Partitioning by channel_id makes a history read (WHERE channel_id = ? ORDER BY ts DESC LIMIT 50) a single-partition range scan, the cheapest thing that engine can do, and the hot path for scrollback.

Channel membership (relational store, joins and constraints, modest size):

• Read state (last_read_ts) sits next to membership, so an unread count is one counter comparison: is last_read_ts less than the channel's latest ts. No scanning messages to count unreads.

Why split the stores.

• Membership is small, strongly consistent, and queried relationally (is this user allowed in this channel).
• Messages are huge, append-only, and queried by time range inside one partition.
• Forcing both into one engine makes one slow: a relational DB chokes on 440 TB/year of append traffic, and a wide-column store cannot do the membership joins and uniqueness constraints cleanly.

Workspace metadata sits in the same relational store keyed by workspace. Nothing exotic.

6Deep dives#

6.1 Fan-out: the central decision#

This is the heart of the problem. When a message lands, how do I get it to every online member? Two ways.

• Fan-out on write (push). On write, resolve the online member set and write the message toward each. Delivery becomes trivial on the read side; writes become expensive.
• Fan-out on read (pull). Persist the message, bump a "channel has new activity" signal, and let clients pull when they next sync. Writes are trivial; the first view is a touch slower.

Naive pure push is fine for a 20-person channel. For a 100,000-member announcement channel it means 100,000 writes for one message, and most of those members are offline anyway. So one policy cannot serve both.

So I split by channel size.

• Small channels (under about 5,000 members): fan-out on write to online members. The fan-out service resolves the online set from the presence store and publishes one event per gateway that holds those members.
• Large channels: fan-out on read. I persist the message, bump a lightweight activity signal, and let clients pull on their next sync. Online viewers of that specific channel still get a push via the gateways subscribed to it, but I never materialize 100,000 copies.

The one optimization that makes it survivable: publish per gateway, not per member.

• If 40 online members sit on 6 gateways, I publish 6 messages, not 40.
• Each gateway locally fans out to its sockets.
• This turns delivery from O(online members) into O(gateways touched), which is bounded by cluster size, not channel size.

The routing lives in a pub/sub topic per channel. When a gateway sees a user connect, it adds that user's channel subscriptions to its local set, and the bus now routes that channel's events to that gateway.

The threshold is a knob, not a law.

• Too low and too many channels go pull, adding a round-trip to busy channels that want to feel instant.
• Too high and a 4,000-member channel during an all-hands hammers fan-out on write.
• Tune it from the real delivery-event rate per channel, and move the line based on load, not just member count.
• A 6,000-member channel that is quiet all day is cheaper to push than a 3,000-member channel mid-incident.

Fan-out across the bus is at-least-once, and that is fine here.

• A retry or a lost ack can deliver the same message twice over a socket.
• No dedup table needed on the server: the client dedupes on message_id, a one-line set-membership check.
• I do not chase exactly-once on the wire, because acks on every hop cost coordination and buy nothing when idempotent client dedupe is nearly free.

6.2 The stateful WebSocket gateway and surviving deploys#

Separate from the fan-out problem is the problem of holding 12 million sockets without losing a message when you ship code.

• The gateway tier is stateful, and stateful tiers are where availability goes to die.
• A client connecting does not care which gateway it lands on. A WebSocket-aware load balancer assigns it to any gateway with capacity.
• The gateway registers "user U9 is on gateway-14" in a fast shared store (Redis), so the fan-out layer knows where to route.

The connection math.

• Each gateway holds maybe 100K to 500K connections depending on message rate, so 12M connections is 24 to 120 gateway processes.
• Memory per connection is the constraint: a few KB of state each, which at 500K connections is a couple of GB of socket bookkeeping per process plus kernel buffers.

The deploy problem, and how I dodge the reconnect storm.

• Restart a gateway and I drop every socket on it: up to 500K people reconnect at once, and that storm can cascade into the load balancer and the other gateways in the same instant.
• Drain gracefully: stop accepting new connections, let the load balancer steer new sockets elsewhere, then close existing ones in waves with jittered backoff hints so clients do not all reconnect in the same 50ms.
• Clients hold an exponential backoff with jitter, so the herd spreads out instead of arriving together.

The gateway holds no durable state. That is the whole trick.

• It is a router. Nothing on it is the only copy of anything.
• Every client tracks the last ts it saw per channel. On reconnect it sends those cursors and the server replays anything newer from the message store.
• A crash is then indistinguishable from a brief network blip on the client's side.
• Delivery is at-least-once over the socket (a message can arrive twice if an ack is lost), and the client dedupes on message_id.

6.3 Presence and typing: cheap, approximate, expiring#

Presence looks easy and is a trap at 12M connections.

• Write every active-to-away transition to a database and fan it out to everyone who can see that user, and presence traffic alone dwarfs messages.
• A user in 50 shared channels toggling active-to-away triggers thousands of notifications for one trivial state change.

A few choices, used together:

• Presence lives in Redis with a TTL. A gateway heartbeats "U9 active" with a 30s expiry. No heartbeat, the key expires, the user is away. No explicit "went offline" event is needed, which matters because a crashed client never sends a clean goodbye.
• Presence is pull-based and subscription-scoped. The client asks for presence of the specific users it is looking at right now (the open member list, the DM it is reading), not all contacts. I am not pushing 12M presence changes to everyone who shares a channel.
• Typing indicators never touch durable storage. Fire-and-forget pub/sub events with a 3-second client-side expiry. If one is lost, the dots just stop a moment early.

The trade-off I am making on purpose.

• Presence is eventually consistent and can be a few seconds stale.
• That is completely fine for "is Sarah online," and it saves the system from drowning.
• I would push back hard on anyone wanting strongly consistent presence: the cost is enormous and no human can tell "online now" from "online 3 seconds ago."

7Bottlenecks and trade-offs#

The fan-out layer is the pressure point.

• Under a thundering-herd event (everyone posts and reacts in one channel during an incident or an all-hands), the pub/sub bus and the gateways carrying that channel get hot.
• Publish-per-gateway keeps bus load proportional to gateway count, so a 100K-member channel costs at most about 120 publishes per message, not 100,000.
• For truly viral channels the read-fan-out path caps the damage, because writes stay O(1) no matter how many members pile in.
• Scale gateways on connection count and per-gateway event rate, not raw bus depth, and partition the bus by channel so one hot channel cannot head-of-line block the rest.

Hot spots in the stores.

• The connection-location store (which user is on which gateway) sits in the path of every fan-out, so it is sharded Redis with replicas.
• Fan-out must tolerate a stale location by failing soft: if the routed gateway no longer holds the user, the message is still durable and the client catches up on reconnect. I never let a stale location entry turn into a lost message.
• A single mega-channel is a hot partition in the message store. For the few giant channels I shard the partition by a time bucket (channel_id plus day), so a year of an active channel does not sit on one node and an all-hands spreads across the current day's range instead of hammering one row range.

Consistency trade-off.

• Messages are strongly ordered within a channel: the monotonic ts per channel partition gives everyone the same order, and clients sort by ts, not arrival time, so even out-of-order socket delivery renders correctly.
• Delivery across channels and presence are eventually consistent. A DM notification might land a half second before the channel message, and presence can lag.
• I traded global ordering (which nobody can perceive) for per-channel ordering (which people absolutely notice when a reply shows above the question it answers).

Single points of failure.

• The load balancer, the pub/sub bus, the connection-location store, and the message store all get clustered and replicated, with no single node able to take the system down.
• The gateway tier is horizontally sharded and holds no durable state, so losing a process loses connections, never messages.
• Reconnect-plus-cursor-replay means a lost gateway rebuilds every client's view from the store.

Observability and SLOs.

• SLOs as measured targets: same-channel delivery p99 under 500ms, and no acked message ever lost.
• Watch: delivery-event rate per channel, bus lag, connection count per gateway, reconnect rate, cache hit ratio for recent messages, write-path error rate.
• When a number moves, I already know which lever it maps to.

Search, briefly (its own subsystem).

• A separate inverted-index service (Elasticsearch-style) fed asynchronously from the write path, so indexing lag never slows a send.
• Sharded by workspace so a query only touches that workspace's shards, with results filtered by channel membership at query time so you never see messages you cannot access.
• Eventually consistent: a message is findable a second or two after it is sent, which is fine.

Where a single policy would have won.

• For a small-team-only product with no giant channels, pure push to online members is simpler and I would skip the read-fan-out path entirely.
• I only pay for two paths because the 100K-member channel exists and would melt pure push.

Always let the biggest channel, not the average one, pick the fan-out policy.

8Summary#

The decisions that matter:

• Decouple send from delivery. The API service only persists and publishes, so send latency is independent of channel size. The single most important seam.
• Size-based fan-out. Push to online members for normal channels, pull for giant ones, because one policy cannot fit a 10-person DM and a 100K announcement channel.
• Publish per gateway, not per member. The one optimization that keeps the bus survivable: delivery scales with gateway count, not member count.
• Stateless-logic gateways with client-side cursors. The gateway is a router, durability lives in the store, and reconnect-plus-cursor-replay makes deploys and crashes non-events. At-least-once over the wire with client dedupe, not exactly-once.
• Expiring, pull-scoped presence, so eventual consistency lands only where users cannot perceive it.
• Per-channel ordering via monotonic ts, eventual consistency everywhere else.
• Out of scope by choice: video calls, the app platform, deep thread fan-out, cross-region routing. Naming them keeps me honest about what 45 minutes covered.

For how specific companies probe this one, see our related guides on the engineering interviews at major real-time and infra companies.