A service with a 200 ms client deadline is offered 3x its capacity for ninety seconds. Its work queue is FIFO and unbounded. Which queue discipline should it use during the overload?

What overload does to a FIFO queue

At 3x offered load the queue grows at 2x service rate. Every item at the head was enqueued longest ago, so under FIFO the work you do first is the work most likely to be already abandoned. With a 200 ms deadline and a queue depth of one second of work, the service is spending 100% of its capacity producing answers nobody is waiting for, and the queue never drains because the useful throughput is zero. The queue has become a buffer for a caller that left.

This is bufferbloat with a different name, and the published fix is the same one. CoDel (Nichols and Jacobson, 2012) does not measure queue length; it measures the time items spend in the queue and acts when the minimum sojourn time stays above a target for longer than an interval. Length is a proxy that varies with service rate; delay is the thing the caller actually experiences.

What each discipline buys and pays

Delay-based FIFO with shedding keeps ordering, bounds the latency of anything it accepts, and converts excess load into fast rejections the caller can retry or degrade on. It pays in rejected requests and a controller with two parameters to tune (a target sojourn time, conventionally a fraction of the deadline, and the interval over which it must persist).

Bounded LIFO is the better-known trick and it is a real improvement on unbounded FIFO: serving the newest item first means the request you serve has the most deadline left, so useful throughput stays positive during overload. Its price is ordering and fairness. Under sustained overload the oldest items are starved indefinitely, and for anything with per-entity ordering requirements that is a correctness problem, not a latency one. It is the right answer when you cannot measure sojourn time cheaply.

Why the other options fail

• Keep FIFO and grow the queue. This is the default and it is the failure. An unbounded queue converts a throughput problem into a latency problem and then into a memory problem, and it guarantees that recovery is slower than the overload: the backlog must be served before any fresh request is.
• Bounded LIFO with shedding. Correct in spirit and second-best here. It fixes the deadline problem without measuring delay, and gives up ordering and tail fairness to do it. Choose it when sojourn time is not available in the queue implementation.
• Priority FIFO by customer tier. Priority is orthogonal to the problem. It decides whose work is thrown away, which is a real and necessary policy, but with no bound and no delay signal every tier's queue still fills, and the top tier simply waits behind other top-tier work.

When this is the wrong answer

With a handful of internal callers that retry politely and no deadline worth defending, a bounded queue of modest depth and a 503 at the limit is enough; the controller is machinery you will not tune. Any delay-based scheme also needs the deadline to be real: if callers wait indefinitely, shedding at 100 ms converts a slow success into a failure, and you have made things worse. Propagate deadlines first, then shed against them.