Design Google Docs (Collaborative Editor)

1Clarifying the requirements#

Pin down scope before drawing a box. The answers here change the algorithm, so treat them as load-bearing. Here is how I would open:

Candidate: When you say "Google Docs," do you want the full suite with comments, suggestions, and spreadsheets, or the core text editor with real-time co-editing?

Interviewer: Core rich-text editor with real-time co-editing. Comments are a nice-to-have, spreadsheets are out.

Candidate: How many people can edit one document at once? That number changes the fan-out design a lot.

Interviewer: Design for up to around 50 concurrent editors on a single doc, but most docs have 1 to 5.

Candidate: Do we need offline editing, where I close my laptop, edit on a plane, and sync later?

Interviewer: Yes. Offline-then-reconnect must not lose edits.

Candidate: Last question. When two edits truly conflict, what's the acceptable outcome? Everyone converges to the same doc, even if it isn't exactly what either person intended?

Interviewer: Correct. Convergence is non-negotiable. Intent preservation is best-effort.

What those answers just told me:

• "Convergence non-negotiable, intent best-effort" is the whole interview handed to me in one line.
• I am allowed to produce a slightly surprising merge. I am never allowed to produce two different documents.
• That single rule is what lets me pick an algorithm later without agonizing over edge cases.

Functional requirements

• Create, open, edit a rich-text document (bold, lists, headings).
• Many users edit the same doc concurrently, changes replicate live.
• Live cursors and selection presence for each collaborator.
• Offline editing, then reconnect and merge without loss.
• Version history, so you can see and restore older states.
• Sharing and permissions (viewer, commenter, editor).

Non-functional requirements

• Edit-to-screen latency under about 200 ms p99 for collaborators in the same region.
• Convergence: all clients on a doc end at byte-identical content.
• Durability: an acknowledged edit survives a server crash, with no silent loss.
• Availability target 99.9%. A doc briefly read-only beats a doc that corrupts.
• Scale to tens of millions of docs and millions of daily active users.

2Back-of-the-envelope estimation#

Get the scale in your head, because it decides the architecture. One number drives everything, and I will flag it when I reach it.

Users and connections.

• 20M daily active users. Say 10% are actively editing in any given minute, so about 2M concurrent editors spread across docs.
• Concurrent open sessions (WebSockets): about 2M live connections during peak.

Edit throughput.

• An active typist produces maybe 3 to 5 operations per second while typing, lower averaged over a session. Call it 1 op/sec/active-editor.
• 2M active editors gives about 2M edit ops/sec at peak. Reads (opening, scrolling) are higher but cheap by comparison.
• Op size: a single-character insert plus metadata is about 100 bytes on the wire after framing.
• 2M ops/sec x 100 bytes = about 200 MB/sec ingest at the edit layer.

Storage (the rounding error).

• We do not store every keystroke forever in the hot path. We snapshot.
• A doc averages maybe 50 KB of content plus a bounded op log.
• 50M docs x about 200 KB (content, history, indexes) = about 10 TB for the corpus, trivially shardable across a handful of machines.

The number that drives everything.

• Not storage, not QPS. It is the p99 latency budget of about 200 ms combined with every op flowing through a single ordering point per document.
• That forces an in-memory, single-owner model per doc: one server owns one doc's live session and serializes its ops.
• You cannot serialize ops through a database round-trip plus cross-region consensus on every keystroke and still hit 200 ms.
• The ordering has to happen in RAM, on one box, per doc. Everything else in this design exists to protect that box and recover it when it dies.

So what shape is this.

• Connection-heavy, latency-bound, ordering-bound. Not a storage problem.
• 10 TB is a rounding error. The hard part is 2M live sockets all demanding a sub-200 ms serialized answer.

3Core APIs#

Keep the REST surface small and boring. The interesting traffic is over WebSocket.

POST /api/docs                      # create a doc, returns docId
GET  /api/docs/{docId}              # load snapshot + current revision number
PUT  /api/docs/{docId}/permissions  # share with user, set role
GET  /api/docs/{docId}/history      # list versions
POST /api/docs/{docId}/restore      # restore a version

The live editing channel is WebSocket messages, not HTTP:

// client -> server: submit an operation
{
  "type": "op",
  "docId": "d_8213",
  "clientId": "c_55",
  "baseRevision": 142,
  "ops": [ { "retain": 10 }, { "insert": "hello" } ],
  "opId": "c_55:991"
}

// server -> client: broadcast a transformed op
{
  "type": "op",
  "docId": "d_8213",
  "revision": 143,
  "ops": [ { "retain": 10 }, { "insert": "hello" } ],
  "author": "c_55"
}

// server -> client: ack your own op with its assigned revision
{ "type": "ack", "opId": "c_55:991", "revision": 143 }

// presence
{ "type": "cursor", "docId": "d_8213", "clientId": "c_55", "range": [10, 10] }

Three fields do the heavy lifting, and an interviewer will ask why they exist:

• baseRevision is what makes transformation possible. The client says "I produced this edit assuming the doc was at revision 142." The server reconciles that assumption with reality, because by the time the op lands the doc might already be at revision 145.
• opId (client id plus a client-local sequence) is the idempotency key. It lets a client safely resend an op after a dropped ack without double-applying it.
• The ops array is a retain/insert/delete delta, not a full document. That keeps the wire payload near 100 bytes and is the natural unit the transform function operates on.

4High-level design#

The write path and read path share the same live session but behave differently. Draw the spine fast, then defend the one seam.

• A client opens a doc. The API layer routes it to the Collab Server that currently owns that doc's session.
• If no server owns it yet, a coordinator assigns one and grants a time-limited ownership lease.
• That server holds the authoritative in-memory document state and a monotonically increasing revision counter.
• All ops for that doc go through that one server. This is the single ordering point from the estimation, and it is the heart of the design.

The op pipeline, in a fixed order.

• Transform the op against any ops the client has not seen yet.
• Apply it to the in-memory state.
• Assign it the next revision number.
• Append it to a durable op log and wait for that append to commit.
• Ack the sender.
• Broadcast the transformed op to every other connected client on that doc.

The order of those steps is not negotiable, and I defend it in the durability deep-dive.

Why the gateway stays dumb.

• The WS Gateway terminates connections and forwards doc traffic to the right Collab Server using a routing table keyed by docId.
• It does connection termination and routing, nothing stateful about the document itself.
• That keeps it cheap to add gateway nodes as connection count climbs.

Why the op log and snapshots exist.

• The op log (a partitioned log like Kafka, partitioned by docId) gives durability and lets us rebuild a session if a Collab Server dies.
• The Snapshot Worker periodically folds the op log into a compact snapshot in the Doc Store, so we never replay a million ops to open a doc.

The common case stays small.

• Read-only viewers connect the same way but only receive broadcasts; they do not submit ops.
• For the typical 1-to-5-editor doc, the whole thing is one Collab Server, one Redis key, one Kafka partition.
• The machinery only spreads out under crashes and scale.

5Data model#

Use a few stores, each picked for its access pattern. Metadata and the op log are different beasts, so I do not force them into one database.

documents (Postgres)

permissions (Postgres)

op_log (partitioned log + cold store)

snapshots (object store, e.g. S3)

• Metadata goes relational because I need foreign keys, transactions on sharing changes, and "list docs I can access" queries.
• The op log is not a relational fit: append-only, read sequentially, keyed by doc. Forcing it into Postgres makes the write path the bottleneck at 2M ops/sec.
• The invariant I care about most: revision is gap-free per doc. If I ever see 142, 143, 145 in the log for one doc, something applied an op without going through the single owner, and that is a corruption bug, not a lost message.

6Deep dives#

6.1 Concurrent edit conflict resolution: OT vs CRDT#

This is the signature hard part, so I spend real time here. If I get only one deep-dive right, it is this one.

The core problem.

• Client A and client B both saw the doc at revision 142.
• A inserts "X" at position 10. B inserts "Y" at position 3. Both send ops tagged baseRevision: 142.
• Apply both naively and the positions are now wrong: B's insert at 3 shifted everything after it, so A's "position 10" no longer means what A thought.
• The two clients diverge, and divergence is the one thing we promised would never happen.

Two families solve this.

Operational Transformation (OT).

• The server keeps the authoritative revision counter.
• When A's op arrives with baseRevision: 142 but the server is already at 144 (B and someone else got there first), the server transforms A's op against ops 143 and 144 before applying it.
• Transformation rewrites A's intent to account for edits it did not know about: "insert at 10" becomes "insert at 11" if an earlier insert pushed things right.
• Then it is applied at revision 145 and broadcast. Clients do the mirror-image transform locally against any in-flight ops they have not had acked yet.
• This is the model Google Docs actually uses.

What is hard about OT.

• The transform function must be correct for every pair of operation types: insert vs insert, insert vs delete, delete vs delete, each with formatting attributes layered on.
• Getting the transform property right (TP1, that transforming an op pair in either order yields the same result, and TP2 if you ever allow peer-to-peer reordering) is where OT implementations go to die.
• I would be honest with the interviewer: this is a small amount of code that is fiendishly easy to get subtly wrong, and it is the code I would unit-test to death with randomized op pairs.
• The upside: ops are tiny, the model is centralized and easy to reason about, and intent preservation is good.

CRDTs (Conflict-free Replicated Data Types).

• Instead of transforming against a central sequence, each character gets a globally unique, densely-orderable identifier (fractional indices, or a position tree like RGA or Logoot).
• You insert "X" between the IDs of its neighbors. Because IDs are unique and order is defined by the IDs themselves, any two replicas that saw the same set of ops converge with no central transform.
• No server-side ordering authority is strictly required.
• The cost: metadata overhead. Every character drags an ID, and deletes usually leave tombstones so concurrent inserts around a deleted character still order correctly. Document size and memory grow, sometimes a lot, for edit-heavy docs.

My call: server-authoritative OT.

• We already have a single owning server per doc for latency, so the central ordering point exists for free.
• OT exploits that for small ops and strong intent preservation.
• CRDTs would be the better pick only if the requirement were true peer-to-peer or no central server.
• At 2M ops/sec and 200 MB/sec ingest, dragging per-character IDs across the wire is a cost I would rather not pay.
• This is a reversible decision tied to the "we have a central server" assumption. Change that assumption and I flip to a CRDT without reworking the rest of the design.

6.2 Cursor presence and offline sync#

Presence is high-frequency but disposable.

• Cursor positions and selections do not go in the op log and are not durable.
• The Collab Server holds live cursor ranges in memory and in Redis (so a reconnect can rehydrate).
• Broadcast them at a throttled rate, maybe every 100 ms coalesced, not per keystroke. A user dragging a selection fires dozens of updates a second and nobody needs every frame.
• On disconnect, a short TTL on the Redis presence key makes the cursor vanish on its own, so I never need an explicit "user left" cleanup path.
• The subtlety that catches people: a cursor at offset 10 must ride the same OT pipeline as edits, or your collaborator's caret drifts. If someone inserts five characters before your cursor, your cursor is now at offset 15, not 10. Cursors ride the transform.

Offline sync is where OT earns its keep.

• While offline, the client keeps editing and buffers ops locally, each tagged with the baseRevision it was created against and a stable opId.
• On reconnect, the client replays its buffered ops to the server.
• The server transforms each one against everything that happened while the client was away, applies them in order, and acks.
• Meanwhile the client transforms incoming remote ops against its still-unacked local buffer. Both sides end at the same document.
• The beauty: offline sync is not a special case. A two-hour offline buffer is just a very long version of the normal in-flight reconciliation.

The gotcha is delete-versus-delete offline.

• If I delete a paragraph offline and you also deleted it, the transform must treat an already-removed range as a no-op rather than deleting adjacent content.
• Get this wrong and the second delete eats the paragraph that followed.
• This is exactly a TP1 correctness case, and it is why I test the transform function harder than any other code in this system.

Idempotency matters here too.

• If the client reconnects, sends its buffer, and the ack is lost in transit, it resends on the next reconnect.
• The server dedupes on opId, so a resent op that was already applied is dropped, not double-applied.
• Without this, every flaky connection would duplicate text.

6.3 Durability, ownership handoff, and the single-owner failure case#

The scary scenario: a Collab Server owns d_8213, has 20 live editors, and crashes mid-session.

What must survive: every op the server acked.

• The pipeline order is deliberate: transform, apply in memory, append to the partitioned op log and wait for the quorum write to be acknowledged by in-sync replicas, then ack the client.
• Ack before the log write is durable and a crash in that window loses an edit the user was told we accepted, which violates the durability promise directly.
• So the log append sits on the critical path. It costs a few milliseconds per op, cheap insurance against a 200 ms budget.
• With a replicated log and acks tied to ISR (in-sync replica) acknowledgement, a single broker loss does not lose committed ops.

Recovery leans on snapshots.

• A coordinator detects the dead server (missed heartbeats), picks a new Collab Server.
• The new server rebuilds the doc by loading the latest snapshot and replaying op-log entries after the snapshot revision.
• This is why snapshots exist: without them, rebuilding a long-lived doc means replaying its entire op history, possibly millions of entries.
• With a snapshot at revision 10,000 and only 200 ops since, recovery replays 200 ops and is done in a blink.
• Clients reconnect through the WS Gateway, which now routes d_8213 to the new owner. They resume from their last-acked revision, replay any unacked local ops (idempotent by opId), and the session is whole again.

The trade-off is a deliberate availability sacrifice.

• Single-owner-per-doc means that during the few seconds of handoff, the doc is briefly unavailable for writes.
• I accept that because the alternative (multiple servers accepting ops for the same doc) reintroduces the exact distributed-ordering problem we worked to avoid, now across machines with network latency between them.
• One owner per doc, protected by a durable log and a time-limited lease, is the clean line.

The lease is the split-brain guard.

• A server that is alive but network-partitioned from the coordinator stops acting as owner once its lease expires.
• A new owner can be appointed without two owners existing at once.

6.4 Version history and restore#

History falls out of the op log almost for free.

• The op log is already an ordered, gap-free record of every applied op, so "version history" is a read over it, not a new system.
• Named versions are just labeled revision numbers: version N points at revision 10,000, version N+1 at revision 12,500.
• Restore is an edit, not a rewind. To restore version N, I compute the delta from the current state back to that snapshot and apply it as new ops through the same pipeline.

Why restore-as-edit matters.

• Treating restore as a normal op keeps the single-owner invariant intact and keeps the log gap-free.
• A collaborator who was mid-type during the restore has their in-flight op transformed against the restore op like any other concurrent edit, so nobody diverges.
• The alternative (mutating state out of band) would be the one path that bypasses the ordering point, which is exactly the corruption class I am guarding against.

7Bottlenecks and trade-offs#

What breaks first: WebSocket connection count.

• At 2M concurrent connections, the gateway tier is the pressure point long before compute on the ops.
• WebSockets are stateful and memory-hungry: 2M of them is a lot of file descriptors and kernel buffers.
• I scale the gateway horizontally and keep it dumb (routing and connection termination only), so it is cheap to add nodes.
• Connection state is sticky per doc via the routing table, so all editors of one doc land on the path to that doc's owner.

Hot doc fan-out.

• A doc with 50 editors means every op fans out to 49 others, so 2M ingest ops can become about 100M outbound messages in the worst mix.
• Within the single-owner model that is fine at 50 editors per doc.
• If the requirement were 10,000 viewers on one live doc (a lecture broadcast, a shared agenda), single-owner broadcast would choke, and I would add a fan-out tier of relay nodes subscribed to the owner, each handling a slice of viewers.
• The interviewer scoped us to 50, so I design for 50 and name the escape hatch rather than over-build it.

The op log as a shared dependency.

• Every acked op writes to it. It is partitioned by docId, so throughput scales horizontally.
• No single doc generates enough ops (50 editors at about 1 op/sec is 50 ops/sec) to hot-spot a partition.
• The risk is a few mega-popular docs landing on the same partition by hash collision. I would monitor per-partition lag and, if needed, give very hot docs their own dedicated partition.

SPOF check.

• The coordinator that assigns doc ownership is a potential single point of failure.
• I run it as a small replicated service with leader election (via the same consensus layer that backs the ownership leases), so losing the leader triggers a re-election, not an outage.
• Ownership leases have TTLs, so a partitioned-but-alive server cannot keep acting as owner after it loses its lease. Same split-brain guard as recovery, viewed from the coordinator's side.

The consistency call I made on purpose.

• Strong, linearizable ordering per document: one owner, gap-free revisions, durable log before ack.
• Weak, best-effort for presence: cursors can be stale or lost and nobody cares.
• Brief per-doc write unavailability during owner handoff, accepted in exchange for never having two owners.
• Strong where correctness lives, loose where it does not, is the design. Spending strong-consistency machinery on cursor positions is a waste; skimping on it for edit ordering is a corruption bug.

Put the strong-consistency money on edit ordering, and let the cursors be sloppy. The user notices a corrupt doc; nobody files a bug about a caret that was 80 ms stale.

8Summary#

The decisions that matter:

• One owning server per doc. A single ordering point is what makes low latency and clean convergence possible. It is the backbone, and everything else protects or recovers it.
• Server-authoritative OT. We already have the central server and ops stay tiny, so OT is nearly free. CRDT is my pick only if the requirement turns peer-to-peer or serverless.
• Durable op log appended before client ack. Acknowledged edits never vanish, with snapshots to keep doc-open and crash recovery fast.
• Presence kept cheap and ephemeral in Redis with a TTL, transformed through the same pipeline as edits so cursors do not drift.
• Offline handled by buffered, base-revision-tagged, idempotent ops that replay and transform on reconnect, treated as just a long version of normal reconciliation.
• Out of scope by choice: spreadsheets, the comments and suggestions layer, the giant-audience read-only fan-out tier, multi-region. Naming them keeps me honest about what 45 minutes covered.

For the real-time delivery and WebSocket gateway patterns here, the Slack system design guide goes deeper on channel fan-out and presence at scale.