Loss-Limited Throughput

A nightly job copies 400 GB over a 10 Gbps path with an 80 ms round trip. A single stream sustains about 5 Mbps. Nothing is misconfigured, no device is saturated, and buying a 40 Gbps link will change nothing.

Loss-based congestion control interprets every lost packet as congestion and halves its window. On a long path with steady low-level loss, the window spends its life being halved and slowly rebuilt, so the achievable rate is a function of the loss rate and the round-trip time. The classical approximation, from Mathis and colleagues in 1997, is:

throughput ≈ MSS / (RTT × sqrt(p))

Link capacity does not appear. That absence is the whole point, and it is why this is the first number to compute when a wide-area transfer underperforms.

Why it matters

It redirects spending. The instinctive response to a slow transfer is more bandwidth, and bandwidth is the one input the formula ignores. Teams buy a larger circuit, see no change, and conclude the problem is mysterious.

It also explains a confusing asymmetry: the identical job is fast inside one data centre and slow across a continent. At 0.5 ms round trip and the same 0.1% loss the bound is roughly 740 Mbps; at 80 ms it is about 4.6 Mbps. A factor of 160 appears from the round-trip time alone, with no change to the code, the hosts or the loss rate.

Implementation patterns

• Compute the bound before investigating anything. With a 1460-byte segment (11,680 bits), 80 ms and p = 0.001: 11,680 / (0.08 × 0.0316) ≈ 4.6 Mbps. If observed throughput matches, the diagnosis is complete.
• Establish whether the loss is congestive. Non-congestive loss (a degraded optic, one bad member of a link bundle, a lossy radio hop) is worked around. Congestive loss means an uplink is full and working around it makes things worse for everyone.
• Parallel streams, when the loss is non-congestive: each stream experiences loss independently, so n streams give roughly n times the bound. 400 GB in four hours needs about 230 Mbps, so around 50 streams suffices. This is why bulk-transfer tools default to many connections.
• Model-based congestion control such as BBR on the sender, which paces to an estimate of bottleneck bandwidth and propagation delay rather than reacting to loss. Appropriate on paths you own end to end.
• Raise socket buffers to at least the bandwidth-delay product - 100 MB in this example - so that configuration is not a second, hidden ceiling once loss is addressed.
• Reduce the round trip where you can by moving the data closer: a relay in the destination region turns one long lossy path into a short one plus a long one with better characteristics.

Industry example

The relation is the reason the scientific-computing and media communities built dedicated bulk-transfer tooling rather than using single-stream copies: parallel-stream and UDP-based transfer tools exist specifically to escape this bound on long paths. It is also why BBR, published by Google engineers in 2016 and widely deployed since, was framed as a congestion-control change rather than a capacity change - the stated target was exactly the long path with non-congestive loss, where loss-based control leaves most of the link unused.

Failure scenarios

• The link upgrade that changes nothing. The most expensive wrong answer, and the most common.
• Buffer tuning with loss present. Hours spent on window sizes while the window is being halved every few round trips.
• Parallel streams over a congested uplink, which converts one team's slow transfer into everyone's packet loss.
• Unilateral BBR on a shared path, which can take more than its share from loss-based flows and turns a transfer problem into a neighbour problem.
• Mistaking the bound for a defect and escalating to the network team, who correctly report that nothing is wrong.
• 0.1% loss accepted as normal when it is one bad path in an equal-cost bundle, and a cable swap would have removed the problem entirely.

Trade-offs

When not to use it

The bound is about a single loss-based stream over a path with meaningful round-trip time. It is the wrong lens inside a data centre, where round trips are sub-millisecond and the constraint is nearly always the application, the disk or the receiver.

It is also the wrong lens for small transfers: a 50 KB API response completes in a couple of round trips and never reaches a steady-state window, so its latency is governed by handshakes, slow start and retransmission timers instead.

And where the transport is QUIC or another protocol with its own loss recovery and pacing, compute the bound only as a sanity check - the controlling behaviour is that stack's, not classical TCP's.

Interview question

Q: A team reports that a nightly 400 GB replication between two regions takes nine hours on a 10 Gbps link and asks you to approve a circuit upgrade. What do you ask for, and what do you expect to recommend?

What a strong answer covers: asking for round-trip time, loss rate and whether throughput is single-stream before anything else · computing MSS / (RTT × sqrt(p)) and comparing it with observed throughput · distinguishing congestive from non-congestive loss and why the recommendation differs · parallel streams sized from the required rate, or BBR on an owned path · socket buffers raised to the bandwidth-delay product so configuration is not the next ceiling · declining the upgrade with the reason that capacity is not a term in the bound · and the cheapest possible outcome, which is finding the single degraded path member responsible for the 0.1%.

Quick check

Quiz: Why does a faster link not speed up a single long-haul TCP transfer with 0.1% loss? Because the achievable rate is approximately MSS / (RTT × sqrt(p)) and link capacity is not in that expression - the window is bounded by loss recovery, not by bandwidth.

Flashcard: Same job, same loss rate: 740 Mbps inside a data centre and 4.6 Mbps across a continent. What changed? - Only the round-trip time, from 0.5 ms to 80 ms, which the bound divides by directly.