A team moves a slow 800 ms enrichment call out of the request path onto a queue. Request p99 drops immediately, total system throughput does not improve, and the time until the user sees the enriched result gets worse. Why does the queue not reduce the work?

The mechanism

A queue moves who waits; it does not change how fast the work gets done. The enrichment service still processes the same number of items per second after the change as before, because its capacity is a property of that service — its CPU, its database, its connection pool — and none of those changed.

The arithmetic is unforgiving. If enrichment can complete 200 items per second and requests arrive at 250 per second, the backlog grows by 50 items every second: 3,000 items after a minute, 180,000 after an hour, and the age of the oldest item grows without bound. Synchronously, the same overload appeared as timeouts within seconds. Asynchronously, it appears as a number on a dashboard nobody is watching, hours later.

What actually improved

Real and worth having: the user's request no longer holds a thread, a connection and a socket for 800 ms. A web tier that could hold 500 concurrent requests now completes each one in 20 ms instead of 820, so it serves roughly 40 times as many requests with the same resources. Availability improved too — an enrichment outage no longer fails the user's request, it delays a result.

What got worse

• Time to the visible result. Completion is now queue wait plus processing, and queue wait is zero only when the system is idle. Under load it is the longest part.
• The failure surface is new and quieter. Errors that used to return to the caller now land in a dead-letter queue, which needs an owner, an alarm and a replay path. Without those, failed enrichments are silently lost.
• Ordering and duplicates. At-least-once delivery means the enrichment must be safe to run twice, which is work that the synchronous version did not require.
• The signal to watch changed. Queue length tells you little; age of the oldest unprocessed item tells you whether you are keeping up, and it is the alert that matters.

When the queue is still the right call

When the caller's outcome does not depend on the result — the user's page is correct without the enrichment — the queue is right even though nothing got faster, because it decouples availability. It is the wrong call when the user is going to sit and watch a spinner for the result anyway, which just relocates the wait and adds machinery.

If the actual problem is that enrichment is too slow, the fix is capacity or a faster enrichment, and the queue only buys time to build it.

Common weak answers

• "Queues improve throughput." They improve the throughput of the component in front of the queue. Total system throughput is set by the slowest stage, and that stage is unchanged.
• "We can scale the consumers." Often true, and it is the fix — but it is capacity that fixes it, not the queue. If the enrichment service is bounded by a database, adding consumers moves the contention.
• "It is now fault tolerant." Buffering during a brief outage is genuine. A queue with no dead-letter handling and no age alert converts visible failures into silent data loss.