Serverless Architecture

1The core idea#

Serverless does not mean no servers. It means you stop owning the box.

• You hand the provider a function or container; they run it when a request shows up.
• They throw away the running copy when traffic dies down.
• You pay per invocation and per gigabyte-millisecond of compute, not per hour of a machine sitting idle.

The mental model I use.

• Traditional servers are renting an apartment: you pay rent whether you sleep there or not.
• Serverless is a hotel that bills by the minute and only unlocks a room when you walk up to the desk.
• Nobody checks in tonight, you pay nothing.

Two flavors matter in interviews.

• Functions as a Service (FaaS): AWS Lambda, Google Cloud Functions. You ship a single function.
• Managed backend services (BaaS): DynamoDB, S3, Cloud Run, managed queues. The whole thing scales and bills on demand.
• Most real serverless systems are a pile of FaaS glued to managed services. The glue is where the design decisions live, and exactly where interviewers probe.

The defining properties you should be able to recite.

• You do not provision or patch servers.
• Scaling to zero and scaling out is automatic.
• Billing tracks actual usage down to the millisecond. That last one is the whole pitch.

2How it works#

Here is what happens when a request lands on a Lambda-style function behind an API gateway.

The gateway maps a route to a function.

• It receives the HTTP request and routes it.
• The platform looks for a warm instance: a running copy of your code in a reused container.
• Warm instance exists: it forwards the event and you pay single-digit milliseconds of overhead.

No warm instance means a cold start.

• The platform pulls your package, spins up a sandbox, loads the runtime, runs your init code, then handles the request.
• That first request takes anywhere from 100ms to several seconds, driven by language and package size.
• A trimmed Node or Go function often cold-starts in 100ms to 400ms.
• A fat Java or .NET function loading a Spring context can hit 2 to 5 seconds.

Scaling is one-request-per-instance in the classic model.

• Ten concurrent requests means ten instances. The platform spins them up and down for you.
• There is a ceiling: AWS defaults to 1000 concurrent executions per account per region, and you have to ask for more.
• Do the math: if each invocation runs 200ms, 1000 concurrent slots caps you near 5000 requests per second before you start getting throttled with 429s.

State is the part that bites people.

• The function is stateless by design.
• Anything in a local variable or on disk might vanish on the next invocation, or survive if the instance gets reused. You cannot rely on either.
• All real state lives in the managed services around it: the database, the cache, object storage, the queue.
• The function is just a short-lived transformer between an event and those services.

3Trade-offs#

This is the section interviewers actually care about. Anyone can define Lambda. Can you tell me what it costs you?

The big win is operational.

• No capacity planning, no autoscaling groups to tune, no 3am page because a node ran out of memory.
• You scale from zero to thousands and back without touching anything.
• For spiky or unpredictable traffic, that is worth a lot.

The big losses cluster around four things.

• Cold starts add latency you cannot fully remove. A 1.5 second cold start on a Java function blows a 200ms p99 budget on exactly the requests that matter. Pay for provisioned concurrency to keep instances warm and you are now paying for idle capacity, which defeats the original pitch.
• Cost inverts at scale. Cheap when idle, expensive when busy. A function running flat out 24/7 can cost several times what a reserved EC2 instance of equivalent power would. There is a crossover point: below it serverless wins, above it you are lighting money on fire. Hard number in section 4.
• Statelessness and limits constrain what you can build. No long-lived connections held in the process. Execution time caps (15 minutes on Lambda). Memory caps. Package size limits. A websocket server or a 2 hour video encode is a bad fit.
• Loss of control means you live inside the provider's sandbox. Limited OS access, their networking rules, their observability gaps. Debugging a mesh of 40 functions is genuinely harder than ssh-ing into one box and reading logs.

My rule of thumb.

• Spiky or event-driven, each unit of work short: go serverless.
• High and steady load, or long-lived connections, or heavy compute: containers or VMs win.

4A worked example: webhook handler, cents or a mortgage?#

Let me make the cost crossover concrete, because this is the calculation that wins the interview.

The setup.

• A webhook handler. Each invocation uses 512MB and runs 200ms.
• us-east-1 Lambda pricing: $0.0000083 per GB-second, plus $0.20 per million requests.
• Per-invocation compute: 0.5 GB * 0.2s * $0.0000083 = $0.00000083.
• Add the request fee of $0.0000002, so about $0.00000103 all in. Call it roughly a millionth of a dollar per call.

Now run three traffic scenarios.

• Low: 1 million calls a month. Compute is 1,000,000 * $0.00000083 = $0.83, plus $0.20 in request fees. Total near $1 a month. A VM to hold this idles 99% of the time and still bills $15 to $30. Serverless wins by 20x.
• Medium: 100 million calls a month. 100,000,000 * $0.00000103 = $103 a month. Compare a c6g.large container running the same workload: about $0.068/hour reserved, roughly $50 a month, and it comfortably absorbs 100M evenly-spread calls. Serverless is already losing on steady load.
• High: 2 billion calls a month, sustained. About 770 requests per second flat. Lambda bill: 2,000,000,000 * $0.00000103 = about $2,060 a month. A pair of reserved c6g.large containers behind an autoscaler handling that steady 770 rps costs maybe $150 to $250 a month. Serverless now costs roughly 10x more.

5Where it is used#

The best fit is glue and bursts. None of these justify a server sitting warm all day.

• A webhook handler Stripe calls a few thousand times a day.
• An image thumbnailer that runs when a user uploads a photo.
• A cron job that runs nightly.
• A fan-out pipeline where one upload triggers transcoding, virus scanning, and metadata extraction in parallel.

Real examples that worked well.

• A company ran all CSV ingestion on functions triggered by S3 events and paid under $40 a month, because traffic was bursty and nothing ran between bursts.
• A startup put their entire low-traffic API on Lambda and paid almost nothing during the year they had no users, then moved the hot paths to containers once a few endpoints started getting hammered.
• That staged approach is the smart one: start serverless, graduate the hot paths.

Serverless also underpins a lot of modern data pipelines.

• Events flow through managed queues and streams.
• Functions do the transforms.
• Managed databases hold the results.
• The whole thing scales with load and you never provision a worker fleet.

Where I would not reach for it.

• A high-throughput, latency-sensitive core service running millions of requests a minute with a tight tail-latency budget.
• The cold starts and the cost curve both fight you there. Put that on containers.

6Common pitfalls and gotchas#

The database connection trap (the classic).

• Each function instance opens its own database connection.
• Scale to 500 concurrent functions and you have 500 connections hammering a Postgres box that tops out around 100.
• Fix: a connection pooler like RDS Proxy or PgBouncer between them, or a database built for it like DynamoDB.
• I have watched this one take down production.

Cold starts on the critical path.

• Teams find their p99 latency is awful and blame the network. It is the cold start.
• Keep packages small, pick a fast-starting runtime (Go and Node start far faster than Java or .NET).
• Use provisioned concurrency only for the few endpoints that truly need it.

Runaway cost from loops and retries.

• A function that calls another function that triggers the first one is a recursion bomb that bills you continuously.
• A misconfigured retry on a failing function can rack up thousands of invocations in minutes.
• Set concurrency limits and dead-letter queues from day one.

Treating it like a monolith.

• Forty functions with tangled dependencies and no clear ownership is worse than one well-structured service.
• Serverless does not excuse bad boundaries.

Observability gaps.

• A single request fanning across six functions and three queues is hard to trace.
• Set up distributed tracing early. Debugging after the fact with scattered log groups is miserable.

Vendor lock-in.

• Your functions bind to the provider's event formats, their IAM, their triggers.
• Moving off is real work. Worth knowing before you build your whole company on one vendor's primitives.

7How it shows up in interviews#

Interviewers probe serverless two ways: sometimes it is the explicit topic, more often you propose it inside a larger design and they poke at whether you understand the consequences.

First, they want you to not oversell it. Watch the contrast.

Weak answer: "We'll use Lambda so it scales infinitely and costs nothing."

Strong answer: "Serverless for the bursty ingestion and webhooks where it scales to zero and runs for about a dollar a month at a million calls. Containers for the steady high-QPS core, because past roughly tens of millions of invocations a month the Lambda bill climbs linearly and overtakes a reserved container fleet, and the cold starts would eat my p99 budget."

• The weak answer asserts magic.
• The strong answer names the crossover and puts a number on it.
• That difference is usually the whole signal.

They will almost always push on cold starts.

• Weak answer hand-waves or says "it's fine."
• Strong answer: "cold starts hit p99, so for the user-facing path I'd use a fast runtime and provisioned concurrency on the hot functions, and accept cold starts on the async background jobs where latency doesn't matter."

Expect the database connection question, because it separates people who have run serverless from people who have only read about it.

• Bring up the pooling problem before they ask and you signal real experience.
• Say it plainly: "500 concurrent functions means 500 connections, Postgres caps near 100, so I'd put RDS Proxy in front or reach for DynamoDB."

They may ask about cost. Show you know it inverts.

• "Serverless is cheapest when idle and most expensive at sustained high load, so I'd watch invocation volume and move anything running near full utilization onto reserved capacity."

Finally, they like the staged-migration answer.

• Start on serverless to ship fast and pay nothing while small, then graduate the hot paths to containers as traffic grows.
• That shows judgment, not dogma.
• The candidates who insist serverless is always right, or always wrong, both lose. The skill is drawing the line in the right place and explaining why.

Serverless is not cheaper or more expensive. It is cheaper idle and more expensive busy, and your job is to know which one you are.

For a worked example, see the System Design Question walkthrough on building an image-processing pipeline, where we wire S3 events, functions, and a queue end to end.