metric

Throughput

The rate of work a system completes per unit of time — and a quantity that trades against latency rather than tracking it.

performancecapacitylatency

The trade is the part worth understanding. Batching, queueing and high concurrency raise throughput and raise latency at the same time; the system does more total work per second while each individual request waits longer. A design tuned for maximum throughput will have poor tail latency, and vice versa.

Little's Law connects them: at a given concurrency, throughput and latency are inversely related. Push utilisation towards 100% and queueing delay grows without bound, which is why systems are run with headroom and why "we have spare capacity, the CPU is only at 85%" is often wrong.

Measure it under a realistic mix, and always alongside latency at percentiles. A throughput number on its own is meaningless — any system can achieve high throughput if you allow unbounded latency.