Term Kind Topic What it is
Collective Communication Cost Allreduce Overhead, Synchronisation Cost at Scale concept Throughput The synchronisation term that determines how far a distributed training or inference job can scale, made of a bandwidth component that is nearly flat in participant count and a latency component that is not.
Goodput Useful Throughput metric Throughput The rate of completed work that someone is still waiting for, which diverges from throughput under overload and is the only rate that tracks what users receive.
Saturation Point concept Throughput The load level beyond which additional demand produces queueing and latency growth rather than additional completed work.
Throughput metric Throughput Work completed per unit time — bounded by the system's narrowest resource, and traded against latency.