Design Dropbox (File Sync and Storage)

1Clarifying the requirements#

Pin down what "sync" actually means before drawing a box, because the answer changes the whole architecture.

• The gap between "upload a file" and "reconcile a 2GB file where two lines changed" is the gap between a weekend project and a real sync engine.
• So I open by nailing down delta sync, collaborative editing, sharing, and the consistency model.

Candidate: Are we syncing whole files, or do we need efficient updates when someone changes 2 lines in a 2GB video export?

Interviewer: Assume real users. A 50MB Keynote where they move one slide should not re-upload 50MB.

Candidate: Good, so block-level delta sync is in scope. Do we support real-time collaborative editing of file contents, like Google Docs?

Interviewer: No. Treat files as opaque blobs. If two people edit the same file, you detect the conflict, you do not merge the contents.

Candidate: That simplifies things a lot. Merging opaque bytes is undefined, so I will do conflict detection, not resolution. What about sharing and permissions?

Interviewer: Yes to sharing a folder with another user, read or read-write. Skip public links and granular ACLs for now.

Candidate: Last one: strong or eventual consistency for the file tree? If I rename a folder on my laptop, when must my phone see it?

Interviewer: The metadata, meaning names, tree, versions, should feel strongly consistent. Seconds of propagation is fine, but no two devices should ever disagree about which version is current.

What that last answer just handed me:

• Metadata is strongly consistent, bytes can lag. That one sentence is the whole ballgame.
• "Opaque blobs" means detect conflicts, never merge. Merging bytes is undefined.
• "Share a folder" with no public links keeps the permission model small.

Functional requirements

• Upload, download, and keep a local folder in sync across many devices per user.
• Block-level delta sync: only changed chunks move over the wire.
• File versioning and history, restore a prior version.
• Share a folder with another user, read-only or read-write.
• Conflict detection when two devices edit the same file offline.

Non-functional requirements

• Durability is the top bar: never lose a file. 11 nines of object durability.
• Metadata strongly consistent. Blocks eventually consistent.
• Sync latency: a change reaches other online devices in a few seconds.
• Availability: 99.9% for the metadata service, higher for reads.
• Efficient on bandwidth and storage, with dedup across the whole user base.

2Back-of-the-envelope estimation#

Get numbers that drive decisions, not numbers for decoration. One split falls out of these, and I flag it when I reach it.

Users and devices.

• 50M DAU. Average user has 3 devices and 40GB stored.
• That is 150M connected devices the notification layer may have to hold.

Storage.

• 50M times 40GB = 2 EB raw, before dedup.
• Cross-user dedup on common files (installers, shared decks, stock assets) typically saves 25 to 40%. Call it 1.3 EB net.

Writes.

• An active user saves maybe 100 file changes/day. 50M times 100 = 5B change events/day, about 58K writes/sec average.
• Peak at 3x = 175K writes/sec. Most are small metadata commits, not full uploads.
• Each change touches on average 2 chunks (we chunk at ~4MB), so about 115K chunk puts/sec average.

Reads.

• Sync is pull-heavy on reconnect. Assume reads are 5x writes, about 290K reads/sec average, mostly metadata polls and chunk gets.

Bandwidth.

• 115K chunks/sec times 4MB = 460 GB/sec of block traffic at peak if every chunk were full.
• Delta sync and dedup cut the real figure by maybe 10x, so plan for 40 to 50 GB/sec egress capacity.

The shape of the load, which is the other load-bearing fact.

• 175K commits/sec but only 115K chunk puts/sec, and most commits touch tiny metadata.
• So the metadata write path, not bandwidth, is where I fight for throughput.
• The system is commit-bound, not byte-bound. Hold that thought; it decides the pathological case in the trade-offs.

3Core APIs#

The client talks to two logical services. Metadata is transactional, blocks are content-addressed.

# Metadata: commit a new version of a file as a list of chunk hashes
POST /v1/files/commit
{
  "path": "work/deck.key",
  "base_version": 41,
  "chunks": ["sha256:9f2a...", "sha256:0b7c...", "sha256:aa31..."],
  "size": 52428800,
  "mtime": 1730600000
}
-> 200 { "version": 42 }
-> 409 { "error": "conflict", "server_version": 43 }

# Block: check which chunks the server is missing, then upload only those
POST /v1/blocks/has   { "hashes": ["sha256:9f2a...", "sha256:0b7c..."] }
-> 200 { "missing": ["sha256:0b7c..."] }

PUT  /v1/blocks/sha256:0b7c...      (raw 4MB body)
-> 201 { "stored": true }

# Sync: long-poll for changes since a cursor
GET  /v1/delta?cursor=eyJ2IjoxMDAyfQ
-> 200 { "entries": [ {"path":"work/deck.key","version":42,"deleted":false} ],
         "cursor": "eyJ2IjoxMDQ1fQ", "has_more": false }

Three choices worth defending:

• Commit carries hashes, not bytes. The client uploads blocks first via /blocks/has then PUT, and only then commits the metadata. Bytes land before the pointer flips.
• Blocks are named by their own content, so uploads are idempotent for free: retrying a PUT of the same hash is a no-op. A client can crash mid-upload, retry the whole batch, and nothing double-counts.
• base_version on commit is the other load-bearing field. It is how I detect concurrent edits without a lock the client has to hold across a slow network.

4High-level design#

The architecture splits cleanly into a metadata plane and a data plane.

• Metadata service owns the file tree, versions, and the chunk list per version. Backed by a sharded relational store, because the access patterns are transactional: commit a version, compare-and-set against a base version, read a consistent folder listing.
• Block service is a thin layer over object storage (S3-class) that stores immutable 4MB chunks keyed by their SHA-256.
• Notification service holds the WebSocket connections and tells online devices "something changed, come pull the delta."
• Dedup and GC pipeline runs offline, reference-counting chunks so orphaned blocks can be deleted safely.

The client is not dumb.

• It runs a local file watcher, a chunker, and a local index DB (SQLite) mapping paths to chunk hashes and versions.
• A sync engine reconciles local state against the server delta.
• A lot of correctness lives here: the client decides what changed, dedups locally, and owns conflict naming.

Write path.

• The client chunks the file and hashes each chunk.
• It asks the block service which hashes are missing and uploads only those.
• It commits the version to the metadata service with compare-and-set on base_version.
• On commit, metadata writes a row to the change log and pings the notification service.
• Bytes land before the pointer flips, so a committed version is never missing its chunks.

Read path.

• The client long-polls /delta with its cursor.
• It gets the list of changed paths and new versions, then fetches the new chunk lists.
• It diffs against its local index and downloads only the chunks it does not already have locally.

5Data model#

Use two stores, each picked for its access pattern. Metadata is relational, blocks are a key-value object store.

files (one row per logical file, current pointer):

file_versions (append-only history):

blocks (reference counting, NOT the bytes):

change_log (drives sync, one row per metadata mutation):

Why SQL for metadata.

• Commits need a real transaction. I do a compare-and-set on current_version, append a file_versions row, bump blocks.refcount, and append to change_log, all atomically.
• That is a textbook ACID transaction. Doing it in an eventually consistent store would re-create conflict handling I get for free from a row lock.
• I shard by namespace_id so a user's whole tree lives on one shard and these transactions stay single-shard, which is what keeps them cheap.

Why object store for blocks.

• 1.3 EB of immutable, content-addressed blobs. No transactions, no mutation, just durable puts and gets.
• This is exactly what object storage is built for, and it gives 11 nines of durability for the part that actually matters: the bytes.

6Deep dives#

6.1 The sync protocol, chunking, and content-addressed dedup (the signature hard part)#

This is the heart of Dropbox, so I spend the most time here. It is also where a passing answer separates from a hand-wave.

• Anyone can say "chunk the file." The interviewer wants to hear why fixed chunks fail and what fixes them.

Chunking: the naive version and why it dies.

• When a file changes, the client splits it into chunks and hashes each one.
• The naive approach is fixed 4MB boundaries.
• Insert one byte at the front and every fixed boundary shifts, so every chunk hash changes, so I re-upload the whole file.
• A 2GB video with a one-byte header change would re-transmit 2GB. That kills delta sync.

The fix is content-defined chunking with a rolling hash (Rabin fingerprint).

• I slide a window over the bytes and cut a boundary wherever the rolling hash matches a pattern, say the low 22 bits are zero. That targets a ~4MB average chunk.
• An insert near the front only changes the one chunk it lands in; the cut points downstream re-align because they depend on local content, not absolute offset.
• So editing one slide in a deck changes maybe 1 or 2 chunks out of hundreds, and a one-byte header change on that 2GB video re-sends one chunk, not 2GB.

Content addressing and dedup. Each chunk is named by sha256(chunk). Two consequences fall out:

1. Dedup is automatic. If you and I both have the same Linux ISO, both upload the same chunk hashes, and the block store already has them. The /blocks/has call returns missing: [] and nothing transfers. That is where the 2 EB to 1.3 EB saving comes from.
2. Uploads are idempotent. A PUT of hash X either stores X or confirms X exists. Retries, duplicate network packets, two devices racing to upload the same chunk: all safe, because the content defines the name.

Here is the full write flow with the dedup check.

The sync loop on the receiving device.

• Other devices long-poll /delta?cursor=X. The cursor is the change_log.seq.
• They get back the changed paths and new chunk lists, diff the new list against what their local index already has on disk, and download only the genuinely new chunks.
• Local dedup matters too: if a chunk already exists locally under a different file, no download.

Conflict handling: optimistic concurrency, not merging.

• The interviewer said files are opaque, so merging is undefined. The client sends base_version with its commit.
• Metadata does a compare-and-set: if current_version != base_version, someone else committed first, so I return 409 with the server version.
• The losing client does not clobber. It keeps the server's version as deck.key and writes its own as a conflicted copy, deck (Alice's conflicted copy 2026-11-03).key.
• Both versions survive, the user decides. This is the honest, boring, correct answer for opaque files, and it is what real Dropbox does.

6.2 Metadata service versus block service, and why they must be separate#

I keep insisting on this split, so let me defend it under fire, because an interviewer will ask "why not one database?"

The two halves have opposite shapes.

• Metadata is small, hot, and transactional. Per file version: a path, a size, and a list of hashes, so kilobytes. For 50M users that is maybe tens of TB, fits in a sharded SQL fleet, and every commit needs a transaction.
• Blocks are enormous, cold, and immutable: 1.3 EB that never needs a transaction, just durable storage.

Jam both into one store and you lose either way.

• Put 1.3 EB into a transactional database and it is absurdly expensive and slow.
• Or give up transactions on metadata and reinvent conflict detection badly.
• Split them and each scales independently: metadata by sharding on namespace_id, blocks by throwing object-storage capacity at the problem with no coordination, because content addressing means no two writers ever conflict.

Garbage collection: the race that will bite you.

• Immutable content-addressed blocks need reference counting. On commit I increment refcount for each chunk. When a version falls out of history (retention expires) I decrement. A chunk at refcount = 0 is a GC candidate.
• The race: a decrement to zero can happen at the same moment another user's commit is about to increment the same hash. Delete immediately and I corrupt their file.

The fix is a grace period with a two-phase delete.

• When refcount hits zero I do not delete, I mark the block pending_gc with a timestamp.
• A sweeper only deletes blocks that have been at zero for, say, 24 hours, and re-checks the refcount under a lock at delete time.
• Any new commit touching that hash clears the pending flag.
• This trades some wasted storage (orphaned blocks linger a day) for correctness, and correctness on durability is non-negotiable.

6.3 Real-time notification and the sync cursor at scale#

Polling every device every few seconds against 150M devices would hammer the metadata DB.

• At 150M devices on a 5-second poll that is 30M metadata queries per second just to say "nothing changed," which is insane.
• So I separate "something changed" from "here is what changed."

Notification service holds the connections, carries no data.

• It keeps persistent WebSocket connections keyed by device, and a map from namespace_id to connected devices.
• When metadata commits, it publishes a tiny message, just namespace_id and the new seq, to the notification layer over an internal message bus.
• The notification service pushes a lightweight "you are behind, cursor is now 1045" to the relevant devices. The device then pulls the real delta from metadata.
• The push carries no file data, so the WebSocket fleet stays cheap and holds state that is cheap to rebuild.

The cursor is the change log, and it makes offline a first-class path.

• The cursor is the change_log.seq, monotonic per namespace. A device stores its last-seen seq.
• On reconnect after a week offline, it sends its old cursor and gets every change since, paginated via has_more.
• Offline-then-reconnect stops being a special case: it is the same pull, just with a staler cursor.
• The log also gives a clean audit trail and makes the sync engine deterministic: given a cursor, the set of changes is fixed.

Trade-off I am making on purpose: the push is best-effort.

• If the WebSocket message is lost, the device is simply stale until its next periodic poll (I keep a slow 60-second fallback poll).
• I do not make notifications reliable, because the change log is the source of truth and the cursor makes a missed push self-healing.
• Building exactly-once delivery into the push layer would add huge complexity to save 60 seconds of staleness. Not worth it.

7Bottlenecks and trade-offs#

What breaks first: the metadata write path.

• At 175K commits/sec peak, a single SQL primary melts. I shard by namespace_id so commits stay single-shard and scale horizontally.
• The risk is a hot namespace: a shared folder with thousands of active collaborators all committing. That shard becomes a hotspot and the monotonic seq per namespace becomes a serialization point.
• Mitigation: cap collaborators per shared folder, and for genuinely huge shared spaces, use a coarser per-file optimistic lock instead of a single per-namespace sequence, so commits to different files do not serialize.

Single points of failure.

• The notification service is a soft SPOF: fully down, sync still works via the 60-second fallback poll, just slower.
• The metadata DB is the hard dependency. I run each shard with synchronous replication to a standby and automatic failover.
• The block store I treat as the durable backbone, replicated across availability zones by the object storage layer itself.

The consistency call.

• Metadata strongly consistent, blocks eventually consistent, on purpose.
• Commits are transactional and compare-and-set, so devices never disagree about the current version.
• A freshly committed chunk might take a moment to replicate across block-store regions, so a download in another region can briefly 404 and retry. I accept that: the metadata says the version exists, the bytes catch up within seconds, and the client retries the GET.
• The alternative, forcing synchronous global block replication before commit, would add seconds of latency to every save for a problem that is self-healing.

Large files and small files.

• A 10GB file is fine: it is just many chunks, uploaded in parallel, committed once.
• The pathological case is millions of tiny files (a node_modules), where metadata commits dominate and each file is smaller than a chunk.
• For those I batch commits and pack many small files' bytes together, so metadata throughput, not bandwidth, becomes the thing I tune.
• This ties straight back to the estimation: the system is commit-bound, not byte-bound, so the tiny-file case is the one that actually hurts.

The system is commit-bound, not byte-bound. Let that pick what you scale.

8Summary#

The decisions that matter:

• Split metadata from blocks. Small, transactional, strongly consistent SQL for the tree and versions; huge, immutable, content-addressed object storage for the bytes. This split is forced by dedup and it is the spine of the whole design.
• Content-defined chunking plus content addressing. Delta sync, cross-user dedup, and idempotent uploads all fall out of one idea: name a chunk by its own content.
• Optimistic concurrency with conflicted copies. Files are opaque blobs, so I compare-and-set on base_version and keep both versions on a race. Merging would be wrong, and last-write-wins would be destructive.
• A per-namespace change log as the sync cursor. Best-effort WebSocket notifications on top, a slow poll underneath, so offline-reconnect and lost pushes are both self-healing.
• Two-phase GC with a grace period. Reference-count immutable blocks without ever deleting a block someone still needs, because durability is non-negotiable.
• Out of scope by choice: real-time collaborative merge, public links, granular ACLs, LAN peer sync, client-side encryption. Naming them keeps me honest about what the time covered.

If you want to see how the metadata-versus-blobs split and real-time push play out in a different product, read the Design Slack guide next.