A dashboard product uses WebSockets for live updates. It works at 5,000 concurrent users and falls apart at 50,000. Diagnose the architectural issues.

Why it breaks at scale and not before

Connections are stateful and pinned. Each WebSocket lives on one server for its duration, so the usual stateless scaling assumptions do not apply. Three specific problems appear together at scale.

Fan-out cost. Publishing to a channel with 50,000 subscribers is 50,000 sends. At small scale this is invisible; at large scale it dominates CPU. Message size matters proportionally, and a per-user personalised payload multiplies it.

Backplane load. With multiple servers, a message published on one must reach clients connected to another — a shared pub/sub layer (Redis, NATS, a managed realtime service). Naively, every server receives every message and filters, so backplane traffic grows with servers × messages rather than with subscribers. Servers must subscribe only to the channels their connected clients need.

Reconnect storms. A deployment or an instance failure drops thousands of connections that all reconnect at once — and each reconnect typically triggers a state resync, which is far more expensive than the steady-state stream. Without jittered backoff this is self-inflicted denial of service.

Fixes

Batch and coalesce. A dashboard does not need every update immediately. Aggregate over 250 ms and send one message. Frequently a 10× reduction with no perceptible difference to users.

Send deltas, not full state, and let clients request a full resync when needed.

Jittered exponential backoff on reconnect, plus resumable streams so a reconnect fetches only what was missed rather than everything.

Drain connections gracefully on deploy — signal clients to reconnect over a window rather than dropping them together.

Reconsider the protocol

If the traffic is genuinely server-to-client only — which for a dashboard it usually is — SSE is the better fit. It is ordinary HTTP, so it passes proxies that mishandle upgrades, it is compressed by existing infrastructure, and EventSource reconnects automatically with a last-event ID, giving resumability for free. That removes a meaningful amount of the client code that is currently failing.

What a strong answer adds

Questioning whether realtime is required at all. Polling every 10 seconds is stateless, trivially scalable, and for many dashboards indistinguishable to the user. Realtime is an architectural commitment, and it should be a requirement, not a default.