Term Kind Topic What it is
Collective Communication Cost Allreduce Overhead, Synchronisation Cost at Scale concept Throughput The synchronisation term that determines how far a distributed training or inference job can scale, made of a bandwidth component that is nearly flat in participant count and a latency component that is not.
Saturation Point concept Throughput The load level beyond which additional demand produces queueing and latency growth rather than additional completed work.