Design Google Drive / Dropbox

1Clarifying the requirements#

Pin down the shape of the problem before drawing a single box. The answers here decide whether you build a photo bucket or a sync engine, and those are completely different systems.

Candidate: Are we optimizing for sync across many devices, or is this mostly a web upload and download store like a photo bucket? Interviewer: Real sync. A desktop client, mobile, and web. Edits on one propagate to the others.

Candidate: Do we sync the whole file every time it changes, or just the parts that changed? Interviewer: Just the parts. A 2 GB file where someone fixes one line should not re-upload 2 GB.

Candidate: Collaboration model. Two people editing the same file, do they get live merging like a doc editor, or last-writer-wins with conflict copies? Interviewer: No live co-editing. If there's a genuine conflict, keep both versions and let the user sort it out. Never silently lose data.

Candidate: Sharing and permissions? Interviewer: Yes. Share a file or folder with specific users, viewer or editor roles. Public links are a nice-to-have.

What those answers just told me:

• The center of gravity is sync correctness and delta transfer, not collaborative editing.
• "Just the parts changed" is the line that forces chunking.
• "Never silently lose data" is the line that forces conflict copies over last-writer-wins.
• I write both of those down as load-bearing constraints before I move.

Functional requirements

• Upload, download, rename, move, and delete files and folders.
• Sync changes across all of a user's devices, near real time.
• Only transfer the changed chunks of a file, not the whole file.
• Keep version history so a user can roll back.
• Share files and folders with other users, viewer or editor.
• Detect conflicting edits and preserve both copies.

Non-functional requirements

• Durability first. We do not lose a user's bytes. Target 11 nines on stored objects.
• High availability for reads and metadata, around 99.95 percent.
• Sync latency under a few seconds for small changes on a healthy network.
• Scale to hundreds of millions of files and tens of millions of daily users.
• Storage cost matters. Deduplicate aggressively.

2Back-of-the-envelope estimation#

Get a feel for the load, because one ratio decides the whole architecture. I'll assume 50 million registered users and 10 million daily active users.

Writes.

• Say each active user saves 40 file changes a day across their devices.
• 10M x 40 = 400M writes/day, divide by 86,400 seconds, about 4,600 writes/second average.
• Peak is spiky, so plan for 5x: around 23,000 write ops/second.

Reads.

• Sync is read-heavy because every device polls or listens for changes.
• Reads (metadata fetches, chunk downloads, status checks) run about 8x writes.
• Roughly 37,000 reads/second average, 180,000 at peak.

Storage.

• Average file size 1 MB after mixing lots of small docs with some large media.
• 400M new versions/day at 1 MB is 400 TB of raw new data daily.
• Two things save us: dedup (most "changes" re-upload chunks we already have, so maybe 15 percent is genuinely new, call it 60 TB/day), and expiring old versions after 30 days on free tiers.
• Net growth lands around 15 to 20 PB/year. This is object-store territory, not a database.

Bandwidth.

• 60 TB/day of real uploads is about 700 MB/s ingest on average, several GB/s at peak.
• Downloads are higher, which is why chunk serving rides a CDN and object store, never our app servers.

Metadata (the number that frames everything).

• Say 50 billion file and chunk records total, each row a few hundred bytes.
• That is low terabytes of metadata. It fits comfortably in a sharded relational or wide-column store if we index well.

Why the ratio is the whole design.

• Confuse the two layers and you either try to put petabytes in a SQL store, or try to do transactional version ordering in S3. Both fail.
• Everything below follows from keeping them apart.

3Core APIs#

Keep two conversations cleanly separate: clients talk to a metadata service for "what changed," and to blob storage (via presigned URLs) for the actual bytes.

POST /v1/files/commit
Authorization: Bearer <token>

{
  "path": "/work/budget.xlsx",
  "chunks": ["sha256:4af1...", "sha256:9b02...", "sha256:7c55..."],
  "size": 2097152,
  "base_version": 41
}

POST /v1/chunks/presign
{
  "hashes": ["sha256:9b02...", "sha256:7c55..."]
}
# returns only the hashes we do NOT already have, each with an upload URL

GET /v1/delta?cursor=<opaque_cursor>
# returns the ordered list of changes since the cursor for this user's namespace

{
  "changes": [
    { "path": "/work/budget.xlsx", "version": 42, "op": "modify" },
    { "path": "/photos/old.png", "version": 7, "op": "delete" }
  ],
  "next_cursor": "c:88213:1730600000"
}

Three design choices worth defending:

• The flow is always ask, upload, commit. Client asks which chunks are missing, uploads only those, then commits the file as an ordered list of chunk hashes. Commit is the only op that changes the authoritative version.
• Presign returns only the missing hashes. The server never asks the client to upload what it already has.
• base_version is the entire concurrency story in one integer. I come back to it in the conflict deep dive; it is what turns a silent overwrite into an honest fork.

4High-level design#

The spine is standard, so draw it fast:

• Clients are the smart part: each watches a local folder, chunks changed files, hashes each chunk, and asks the server which are new.
• That keeps our servers out of the heavy byte path, the only way the bandwidth numbers close.

The metadata service owns the source of truth.

• The folder tree, file versions, the chunk list per version, and permissions.
• It writes to the metadata DB and caches hot folder listings in Redis.

The chunk service keeps bytes off our servers.

• It hands out presigned URLs so clients upload and download chunks directly to the object store (S3-style).
• Reads go through a CDN, so a shared file downloaded by 100 people does not hit origin 100 times.

The notification path is a nudge, not a payload.

• When a commit lands, the metadata service drops an event on the notification queue.
• The sync notifier holds long-lived connections (WebSocket or long poll) to online devices and pushes "something changed, come fetch the delta."
• Offline devices catch up with the delta cursor when they reconnect.

The dedup index is the storage saver.

• It maps chunk hash to stored location, so if a chunk already exists anywhere, we never store or transfer it again.

5Data model#

Metadata lives in a sharded relational store, sharded by user or workspace namespace so one user's tree stays on one shard.

• Co-location is deliberate: a commit only ever touches one namespace, so it is a single-shard transaction with no cross-shard coordination.

files (one row per logical file, the current pointer):

file_versions (append-only, never mutated):

chunks (global dedup table):

shares:

The one choice to defend: file_versions is append-only.

• We never overwrite a version. A change is a new version row pointing at a new chunk recipe.
• That gives free version history and makes rollback a single pointer update on files.current_version.
• The metadata defining a version is immutable the moment it is written, which matches the immutability of the chunks it points at.

6Deep dives#

6.1 Chunking and dedup: how we avoid re-sending bytes#

When a file changes, we must figure out which parts actually changed.

Fixed-size chunking is the trap.

• Insert one byte at the top and every fixed 4 MB boundary shifts, so every chunk hash changes and you re-upload the whole file.
• That is the boundary-shift problem, and it quietly destroys the 85 percent dedup savings the estimate counted on.

So I use content-defined chunking.

• Run a rolling hash over the file and cut a boundary whenever the hash hits a pattern (say the low 13 bits are all zero).
• Boundaries tie to content, not offset, so inserting a byte only changes the one chunk around the edit.
• Average chunk lands near 4 MB, with min and max caps so a file that never trips the pattern (or trips on every byte) cannot produce pathological sizes.

Dedup falls out of hashing.

• Each chunk gets a SHA-256. The client sends the list of hashes, the server checks the dedup index, only truly new chunks upload.
• This is exactly where the 85 percent savings comes from.
• If two users save the same PDF, we store it once and bump a ref count.

The subtlety that trips people: commit must be atomic with the ref-count bumps.

• Bump ref counts but the commit fails: you leak references and chunks never get collected.
• Commit but miss a bump: garbage collection can delete a chunk a live version still points at.
• Fix: wrap the commit and the ref updates in one transaction on the metadata shard, which is possible precisely because they share a namespace shard.
• Garbage collection for orphaned chunks (ref_count hits zero) runs as a slow background sweep, never inline, because deleting storage keys is expensive, irreversible, and can safely wait.

6.2 Sync and conflict handling#

This is the signature hard part of the whole question, so I spend the most time here.

The cursor is the spine of sync.

• Every device keeps a cursor, an opaque marker of the last change it saw in its namespace.
• When online, the sync notifier pushes a nudge on any commit; the device then calls GET /delta?cursor=... and applies changes in order.
• When offline, it replays from its last cursor on reconnect.
• The cursor makes sync resumable and idempotent: applying the same delta twice is a no-op because versions are monotonic, so a device already at 42 just ignores 42.

Now the conflict, and this is where base_version earns its place.

• Device A and device B both edit budget.xlsx while B is offline. Current version is 41.
• A commits with base_version 41, server sees 41 equals current, creates version 42.
• B comes online and commits with base_version 41, but the server is already at 42.

The server's rule: never lose data, so fork instead of overwrite.

• B's base is stale (41 against a current of 42). This is a genuine conflict, not a lost race we can retry, because B edited a version that no longer represents the truth.
• The server accepts B's bytes but writes them as a conflict copy: budget (conflicted copy from B's laptop).xlsx, a brand new file with its own version 1.
• Both versions now exist and sync to every device; the user resolves it by hand.
• This is optimistic concurrency control: fast common path, and the compare on base_version turns a silent overwrite into an honest fork.

6.3 Notifications at scale#

The notifier holds tens of millions of long-lived connections.

• 10 million daily users times multiple devices each. One box cannot do that, and one box is a single point of failure for every user's sync.
• Shard notifier nodes by namespace so all of a user's devices land on the same node (or a small consistent-hashed set), keeping the fan-out for any commit local to one node.
• A commit event carries only the namespace_id and a bump signal, not the file payload, so each push is a few bytes. The device pulls the actual delta over plain HTTP.

A notifier outage degrades latency, it never loses a change.

• If a node dies, devices reconnect, land on a new node via the hash ring, and immediately call delta with their cursor.
• They miss nothing, because the cursor (not the live connection) is the source of truth.
• The push is purely an optimization to avoid polling. An outage drops sync from seconds to a poll interval, nothing more.

6.4 Sharing and permission checks on the read path#

Sharing sounds simple until every chunk download needs an authorization decision, at 180,000 reads/second peak.

• Walking the folder tree to the root on every read to resolve inherited permissions would hammer the metadata DB.

So resolve shares once at grant time and materialize them.

• When a folder is shared, write shares rows for the folder and mark descendants' effective access with inherited true.
• Cache a per-user access set in Redis keyed by namespace. A download then checks a cached set, not a tree walk.
• On revoke, invalidate that cache entry and let presigned URLs expire on their short TTL, so revocation is bounded by the URL lifetime: not immediate, but fast and predictable.
• Public links are just a share grant to an anonymous principal with a viewer role, which is why they were cheap to add as a nice-to-have.

7Bottlenecks and trade-offs#

The metadata DB is the real bottleneck, not storage.

• Bytes scale out trivially on object storage.
• Metadata has hot shards: a company with a 50,000-file shared folder means thousands of users hammering one namespace.
• Shard by namespace, and for huge shared spaces give the shared folder its own namespace so access does not stampede a single user's shard.
• Cache hot folder listings in Redis with event-driven invalidation on commit, so a listing is recomputed only when it actually changed.

The dedup index is a shared hot spot.

• Every upload reads it. It is a giant hash-to-location map.
• Keep it as its own horizontally partitioned key-value store, partitioned by hash prefix, so load spreads evenly by content with no natural hot key.
• Ref counts are the risky part: undercount leaks storage, overcount can delete a chunk someone still needs.
• Never garbage collect inline. Use mark-and-sweep that double-checks ref_count is still zero under a lock before deleting, with a grace window so a racing commit can win the chunk back before it is swept.

Consistency model.

• Metadata is strongly consistent within a namespace because commits are transactional on one shard. Version 42 means one specific thing to every device.
• Across namespaces and for the CDN-cached byte layer, we accept eventual consistency.
• That is safe because chunks are immutable and content-addressed: a cache miss or a slightly behind replica still serves correct bytes, because an immutable blob can never be stale.

Small files kill you.

• Millions of tiny 2 KB files mean per-file metadata rows and round trips dominate, not bytes.
• Batch small-file commits and pack tiny chunks together so the object store is not flooded with micro-objects, which most object stores charge for per request.
• The cost model there is request count, not gigabytes, so the fix is amortizing requests.

Thundering herd on a viral shared link.

• A public link going viral is a pure CDN problem. The object is immutable, so the CDN absorbs it.
• Origin sees one miss per edge region, not a million.

The trade-off I would defend hardest: chunk and hash on the client.

• It pushes CPU to user devices and trusts clients to hash honestly, so the server must still verify a chunk's content matches its claimed hash on receipt.
• Skip that and a malicious client could upload garbage under a popular chunk's hash and poison a dedup entry other users then read.
• But client-side chunking moves the expensive work off our servers and is the only way to hit the bandwidth numbers from Section 2.
• The rule: verify the hash on upload, trust it on read.

Win this question on the metadata and sync design, not on storing bytes.

8Summary#

The decisions that matter:

• Split the metadata path from the data path. Metadata service owns truth; clients upload bytes straight to object storage via presigned URLs through a CDN. This falls directly out of the byte-to-metadata ratio.
• Content-defined chunking plus global dedup. Only truly new chunks ever move, which is what makes delta sync and the storage math work.
• Append-only versions. A change is a new version pointing at a new chunk recipe, which gives version history and rollback for free.
• Cursor-based sync with push as an optimization. The cursor is the source of truth, so sync survives dropped connections, node failures, and duplicate deltas.
• Optimistic concurrency with conflict copies, never silent overwrite. A stale base_version forks the file instead of losing an edit. Correctness beats cleverness for binary files.
• Strong consistency inside a namespace, eventual everywhere else. Immutable content-addressed chunks make the relaxed parts safe.
• Out of scope by choice: live co-editing, full-text search, trash retention policy tuning. Naming them keeps me honest about what 45 minutes covered.

For more on the moving pieces here, see our related guides on designing a CDN, a notification and pub/sub system, and a distributed object store.