A grocery platform's retailer inventory API is slow but not failing - responses take 6 seconds instead of 200 ms, and mostly succeed. Should the circuit breaker open? Analyse the trade-off.

Why this case is harder than an outage

A circuit breaker keyed on error rate does nothing here, because the dependency is succeeding. Yet 6-second responses are arguably worse than failures: they hold connections, threads and memory across the whole caller fleet, and they blow the caller's own latency budget while consuming its capacity.

Slow dependencies cause outages more often than failed ones, because failure is loud and slowness is polite.

The case for opening

• Latency is a failure when it exceeds the caller's deadline. A response arriving after the user's page has given up is indistinguishable from an error, but costs far more.
• Resource protection. Every in-flight call occupies a slot. At 30x normal latency the caller needs 30x the concurrency for the same throughput, and will exhaust its pool.
• It stops the amplification. A caller that fails fast stops adding load to the struggling dependency and gives it room to recover.

The case against opening

• The data is genuinely valuable. For inventory specifically, the fallback — last-known stock — is meaningfully worse for the customer. Substitutions and refunds have real cost, so trading freshness for latency is not free.
• Opening on slowness can be self-fulfilling. If the slowness came from a transient spike, cutting all traffic and then restoring it can produce oscillation.
• Partial capacity is still capacity. Refusing all calls when the dependency can serve some of them wastes what is available.

The resolution

Not a binary breaker. A latency-aware, tiered response:

1. Concurrency limiting rather than tripping. Cap in-flight calls to the dependency. Excess calls do not wait — they take the fallback immediately. This preserves resource protection while still using the dependency's real capacity, and it self-adjusts as latency changes.
2. Trip on latency, not just errors. The breaker's signal is "p95 exceeds the deadline for N consecutive windows", not merely error rate.
3. Differentiate by call criticality. Checkout re-validation still calls the dependency and waits, because correctness there is worth the latency. Browse and search take cached stock immediately. This is bulkheading by request class, and it is the highest-value part of the design.
4. Widen the substitution allowance while degraded, so the business consequence of stale inventory is absorbed by the product rather than by a failed order.
5. Half-open with a single probe, so recovery is tested by one request rather than by the fleet.

The lesson

The question "should the breaker open?" is usually the wrong question. The right one is "which callers can afford to wait, and what does each do when it cannot?" A single global breaker forces one answer on operations with completely different tolerance for staleness — and it is precisely the most important operation, checkout, that a global breaker would wrongly cut off.