Term Kind Topic What it is
Caching Strategy pattern Performance & Capacity The chosen pattern for how a cache is populated, read and invalidated — cache-aside, read-through, write-through or write-behind.
Concurrency concept Performance & Capacity The number of operations in progress at once — distinct from parallelism, which is how many are literally executing simultaneously.
Connection Pool concept Performance & Capacity A fixed set of reusable database connections shared by an application's requests, and one of the most common hidden capacity ceilings.
Google Maps and Planetary-Scale Spatial Serving case-study Performance & Capacity Map serving is fast because almost nothing is computed on request — the world is precomputed into a pyramid of tiles, and space is indexed onto a one-dimensional curve.
Hedged Request Request Hedging, Tied Request pattern Performance & Capacity Sending a duplicate of a request to a second replica after a short delay and using whichever response returns first, to cut tail latency.
Horizontal vs Vertical Scaling Scale Out vs Scale Up concept Performance & Capacity Adding more machines versus making one machine bigger — and the fact that vertical is underrated for stateful tiers.
Little's Law concept Performance & Capacity In a stable system, the average number of items in it equals the arrival rate times the average time each spends in it — L = λW.
Little's Law Applied to Pools practice Connection Pooling Using L = λW to size connection and thread pools from measured throughput and latency rather than from a default.
Load Testing practice Performance & Capacity Driving a system with realistic traffic at a target volume to verify it meets its performance targets before real users do.
Tail Latency p99, p999 metric Performance & Capacity The latency experienced by the slowest small percentage of requests, which is what users and dependent services actually feel.
Throughput metric Performance & Capacity The rate of work a system completes per unit of time — and a quantity that trades against latency rather than tracking it.
Twitter's Timeline Fan-Out case-study Performance & Capacity Twitter precomputes each user's timeline at write time but handles very-high-follower accounts at read time, because neither strategy alone survives both ends of the distribution.
WhatsApp's Small-Team Scale case-study Performance & Capacity WhatsApp served hundreds of millions of users with a few dozen engineers by matching one technology choice precisely to the workload and refusing to add anything else.