Performance & capacity Simulation Intermediate 115 parts 15 Oct 2026

Anatomy of a web stack under load

Every request a steel ball. Turn the traffic up until the queues fill, the p99 explodes and the load balancer starts refusing.

Capacity planning is usually taught as formulas: utilisation, Little's law, Erlang C, the hockey stick. This model makes them physical. A discrete-event simulation of a load-balanced app tier, a cache and a pooled database moves every ball you see, and a second copy runs out of sight at full speed so its averages settle in seconds. The page checks the simulation against queueing theory live, shows why the tail degrades long before the mean, and lets you run the failure scenarios architects plan for: a traffic spike, a lost node, a cold cache, and life at 95% utilisation.

The machine

A marble run. A hopper releases balls at the arrival rate; they wait in a trough (the queue, one slot mark per waiting place), a distributor hands each to a free app-server drum that holds it for its service time, and it either drops through the cache unit or rolls into a second trough to wait for one of the database pool's drums. Finished balls land in a tray; when a trough is full the next ball is refused into the reject bin, the 503. Two dials read p99 latency and app utilisation.

What is real. Every ball is a request in an event-driven simulation with Poisson arrivals and exponential service (two M/M/c tiers with bounded queues), checked before the page is built: over long runs its mean latency matches Erlang C within 6% and its queue lengths satisfy Little's law within 1%. The dials, charts and table read the simulation; the theory lines are computed from the same parameters.

What to take away

  • Latency is flat until utilisation nears capacity, then turns up hard: for one server the response time is s/(1−ρ), so 80% to 95% busy multiplies it by four.
  • The tail degrades first: the spare capacity cμ−λ sets how fast long waits die off, so p99 alarms long before the mean looks wrong.
  • Little's law, L = λW, holds for any stable system and sizes pools, threads and buffers without any distribution assumptions.
  • A cache is capacity: the database sees only λ(1−h), so a cold cache can push a healthy database past saturation.
  • A bounded queue that refuses fast keeps the requests it accepts healthy; an unbounded one turns overload into timeouts for everyone.

Sources