Long Polling, SSE and WebSockets

1The core idea#

Start from what HTTP cannot do, not from the fancy option. Every one of these patterns exists to solve the same gap.

• The problem is server-initiated data. Plain HTTP is client-asks, server-answers, connection-closes. Fine for a product page. It breaks the instant the server has news you did not ask for: a new chat message, a price tick, "your driver is 2 minutes away."
• The three patterns are three ways to fix that. They differ in how long the connection stays open and how many directions it carries.

A mental model that sticks:

• Long polling is calling a shop and the clerk saying "hold on, let me check" and not hanging up until there is an answer. One answer per call.
• Server-Sent Events (SSE) is a one-way radio station. The server talks, you listen, over one connection that stays open. You cannot talk back on that channel.
• WebSockets is a phone line both sides can speak on at any time. Full duplex, low overhead per message, until someone hangs up.

2How it works#

Three mechanisms, same TCP underneath. The difference is what happens to the connection after the first request, not some exotic networking magic.

Long polling.

• The client sends a request. The server does not reply "nothing new." It holds the request open, often 20 to 30 seconds.
• The instant data arrives, it responds. The client reads it, then immediately fires another request.
• If the hold times out with nothing, the server returns an empty response and the client reconnects anyway.
• It is still plain HTTP, so it works everywhere: proxies, old browsers, corporate firewalls, all fine.

SSE.

• The client opens one HTTP request with Accept: text/event-stream. The server never closes it.
• The server writes events down the pipe as plain text, one chunk at a time, each one looking like data: {...}\n\n.
• The browser's EventSource API handles parsing and automatic reconnection, sending a Last-Event-ID header so the server can replay what you missed.
• Still HTTP, still one TCP connection, but data flows server to client only.

WebSockets.

• The client sends an HTTP request with an Upgrade: websocket header. The server agrees, and that same TCP connection switches protocols.
• From that point it is no longer HTTP. Both sides send framed messages whenever they want, a few bytes of overhead per frame instead of a full set of HTTP headers.
• It stays open until one side closes it or the network dies.

In production all three go through a load balancer and TLS. The box that actually decides your scale story is behind the service, not the wire.

3A worked example with real numbers#

Abstract talk about "overhead" never lands in an interview. Numbers do. So price out one concrete scenario and show how the three choices diverge.

The setup.

• 100,000 concurrent users watching a live notifications feed.
• Each user gets 1 real update every 30 seconds on average.
• Latency target: fresh updates should feel instant, so under 1 second.

Option A, naive polling every 1 second.

• Each client fires a request per second whether or not anything changed: 100,000 requests per second hitting the fleet.
• Only 1 in 30 carries a payload. The other 29 come back empty, so roughly 96,700 requests per second are pure waste.
• A request plus response with headers and TLS resumption is around 800 bytes to 1 KB even when the body is empty, so you burn on the order of 80 to 100 MB/s to deliver almost nothing.
• Average event latency is half the poll interval, about 500 ms of built-in lag before the server even gets a chance to answer.

Option B, long polling with a 30 second hold.

• A client only re-requests after it receives a reply or the hold times out.
• With one real event every 30 seconds, each client makes about 1 request per 30 seconds: roughly 3,300 requests per second across the fleet, a 30x reduction from naive polling.
• Latency on a real event is near zero, because the request was already parked waiting.
• The cost: 100,000 requests still cycle through the load balancer every half minute, plus thundering-herd risk if a deploy drops them all and they reconnect in the same second.

Option C, SSE or WebSockets, one open connection per user.

• Zero request churn. The 100,000 connections sit idle, and the server writes down each pipe only when there is something to say: about 3,300 messages per second across the fleet, each a few hundred bytes with no repeated HTTP headers.
• Latency is effectively network round-trip, call it tens of milliseconds.
• The cost moves to memory and file descriptors: 100,000 open sockets at a rough 30 to 60 KB of kernel plus app buffer each is 3 to 6 GB of RAM just to hold connections that are idle most of the time, spread across however many servers you shard onto.

The same scenario in one view:

4Trade-offs#

This is the part interviewers actually grade. Compare them on the axes that change your answer.

What long polling costs you.

• Every cycle is a fresh HTTP request with full headers, a new connection setup if keep-alive drops, and a reconnect right after each reply.
• Under heavy load you get a thundering herd when many clients reconnect at once.
• Latency is lumpy: an event that lands right after a timeout waits for the next cycle.
• Pick it when you need HTTP compatibility above all, or updates are rare.

What SSE costs you.

• It is text only and one directional. If the client needs to send frequent messages, you now run SSE down plus regular POSTs up, which is two mechanisms.
• The classic gotcha is the HTTP/1.1 six-connections-per-domain browser limit, which SSE connections eat into. HTTP/2 mostly fixes that with multiplexing.
• Upside that people forget: one connection, auto-reconnect, event IDs for replay, dead simple on the client. Often the right answer.

What WebSockets cost you.

• You own reconnection, heartbeats, backoff, message ordering, and auth refresh.
• Some corporate proxies still mangle the upgrade.
• Because it is not HTTP after the handshake, a lot of normal tooling (caching, standard logging, some WAFs) does not apply.
• The right call when both sides talk often and latency matters, and overkill when they do not.

One thing true for SSE and WebSockets both: each client holds an open connection. Ten thousand idle users means ten thousand open sockets and the memory that comes with them. Long polling trades that for request volume instead. Neither is free.

5Where it is used#

Map each pattern to what real systems actually run on it.

Long polling.

• Older chat widgets, some webhook-style integrations, and a fallback tier when WebSockets fail.
• Many realtime libraries try WebSockets first and quietly fall back to long polling behind a hostile proxy. That fallback is the whole reason long polling is still worth knowing.

SSE.

• Anything that is a stream of server updates with little client chatter.
• A live sports score page. A stock ticker or crypto price feed where you only consume. Build and deploy logs streaming in a CI dashboard. Notification bells.
• Streaming tokens from an LLM API to a browser, which is almost always SSE: one-way server-to-client text, reconnection handled for you.

WebSockets.

• Genuine two-way, high-frequency interaction.
• Group chat and DMs. Multiplayer game state. Collaborative editors where every keystroke from every person has to reach everyone else.
• Live trading terminals where you stream prices down and fire orders up on the same hot path. Presence indicators ("three people typing").

If the client mostly listens, start with SSE and only upgrade to WebSockets once you can name a concrete reason the client must push often.

6Common pitfalls and gotchas#

The transport is the easy 10%. These are where interviews are won or lost.

Treating the transport as the hard part.

• It is not. The hard part is fan-out. When user A sends a message, how does it reach user B whose socket lives on a different server instance?
• You need a pub/sub layer (Redis pub/sub, Kafka, NATS, a managed bus) so any server can publish an event and every server holding a relevant connection gets it.
• Draw WebSockets but no backplane and a sharp interviewer asks "and how does server 2 know?" You will have nothing.

Sticky sessions and scaling.

• An open connection is pinned to one server. Deploy new code and you drop every connection at once.
• Plan for mass reconnect, graceful draining, and a load balancer that understands long-lived connections.
• Autoscaling on CPU is misleading here, because an idle socket costs memory, not CPU.

Idle timeouts killing you silently.

• Load balancers and proxies close connections that look idle, often at 60 seconds.
• For SSE and WebSockets you send periodic heartbeats (a comment line for SSE, a ping frame for WebSockets) to keep the path warm and to detect dead peers.

Auth on a long-lived connection.

• You authenticate at the handshake, but tokens expire. A connection open for hours outlives a 15-minute JWT.
• You need a plan: re-auth messages over the channel, short-lived connection tokens, or forcing periodic reconnect.

Backpressure.

• A slow client cannot keep up with a fast server. Keep buffering events for it and you leak memory.
• Decide early: drop, coalesce (send only the latest state), or disconnect the slow consumer.
• Tie it to the worked example: on the 3,300 messages per second feed, an unbounded per-connection buffer is how one slow phone on a bad network takes down a server.

Assuming WebSockets are always faster.

• For rare updates they are not meaningfully faster, and they cost you an always-open connection plus all the reconnection logic.
• Do not pay for a phone line to receive one message a day.

7How it shows up in interviews#

Interviewers use this topic to see if you reason about trade-offs or just pattern-match. Here is what they probe and how I would answer.

Interviewer: The client needs live updates. What do you use?

Candidate: Depends on direction and frequency. If the client mostly receives, SSE: one connection, built-in reconnect, simple. If both sides send often, like a chat or a game, WebSockets. If I need to survive hostile proxies or legacy clients, long polling as a fallback.

• The weak version is "WebSockets," full stop. It tells the interviewer you know one tool and reach for it every time, and it invites "why not SSE?" which you then cannot answer.
• The strong version names all three and a decision rule, and preempts the follow-up instead of walking into it.

Interviewer: How does this scale to a million connections?

Candidate: Connections are stateful and pinned, so I shard clients across many connection servers, put a pub/sub backplane between them for fan-out, use sticky routing, send heartbeats, and plan for reconnect storms on deploy. A million idle sockets at roughly 30 to 60 KB each is 30 to 60 GB of RAM across the fleet plus a million file descriptors. This is a memory and fd-limit problem, not a CPU one.

• The weak version is "add more servers behind a load balancer." True and useless, because it ignores that the connections are stateful.
• Landing the specific number is what separates "read a blog post" from "ran this in production."

Interviewer: Why not just poll every second?

Candidate: At scale that is a request per client per second whether or not anything changed. 100,000 users is 100,000 requests per second, about 29 of every 30 empty, so roughly 96,700 wasted per second, and you still eat about 500 ms of average latency. Long polling holds the request and cuts that to about 3,300 per second. SSE and WebSockets keep the pipe open and drop steady request churn to zero.

Interviewer: Walk me through a chat app.

Candidate: WebSockets for the live messaging, a pub/sub backplane (Redis or Kafka) for cross-server fan-out, a durable store for message history and offline delivery, presence via heartbeats, and long polling as a fallback. The message gets written to durable storage first; the live socket is a fast-path optimization, not the source of truth.

• Weak candidates stop at "WebSockets connect the users" and cannot explain cross-server delivery or the offline case.
• The offline case is the tell: a strong answer treats durable storage as the source of truth and the socket as a speed boost on top.

Next, try the System Design Question walkthrough on designing a real-time chat system, where these transport choices meet fan-out, presence, and offline delivery head-on.