WebSockets

Definition

A WebSocket upgrades an HTTP connection into a long-lived bidirectional channel. The server can push without being polled, and the per-message overhead is small compared with an HTTP request.

What it changes architecturally

The important consequence is that your stateless service tier becomes stateful in one specific way: a connection is bound to one server process for its lifetime. That has consequences that surprise teams used to stateless request handling.

• Deployments disconnect everyone. Every rolling deploy severs every connection on the replaced instance. Clients must reconnect with exponential backoff and jitter, or the reconnection is a self-inflicted thundering herd.
• Load balancing is by connection, not by request. A newly added instance receives no traffic until clients reconnect, so a scale-out event does not relieve load on existing instances.
• Routing a message to a user requires knowing which server holds their connection. This is the central design problem: either a shared registry of connection locations, or a message bus that every server subscribes to and filters.
• Capacity is measured in concurrent connections and memory per connection, not in requests per second.

Industry example

Slack's real-time delivery makes the shape concrete. A message must reach every currently connected member of a channel within a second, and be available to those who reconnect hours later. Those are two different systems: live fan-out over persistent connections, and durable storage with an efficient catch-up protocol.

Three consequences follow. Connections are sharded so members of a channel are reachable from a bounded set of servers rather than requiring a global broadcast. Presence is its own subsystem — high write volume, low value per write, and explicitly allowed to be approximate, because nobody is harmed by a green dot three seconds stale. And very large channels are treated as a special case, because designing everything for the 100,000-member channel would make the three-person case absurdly expensive.

The catch-up protocol deserves emphasis: after any network blip, thousands of clients reconnect at once. If reconnection means "send me everything since my last cursor" and that is expensive, the recovery becomes a second outage.

Failure scenarios

• Reconnect storms after a deployment or a network event, with no jitter.
• Intermediaries dropping idle connections — proxies and firewalls commonly close after 60 seconds — requiring application-level heartbeats.
• Memory per connection unmeasured, so an instance holds far fewer connections than assumed.
• No backpressure toward a slow consumer, so the server buffers unboundedly and runs out of memory.
• Assuming delivery. A connection can be open while messages are undelivered. Acknowledgement and a resume cursor are required for anything that matters.

Interview question

"A deployment drops 200,000 WebSocket connections. Describe what happens next and how you would make it safe."