At 150% of capacity, what should your service do? Describe the mechanisms.

What is being tested

Whether you can specify graceful overload behaviour, which does not emerge on its own and must be built.

What it should do

Serve 100% of capacity normally and reject the excess quickly and clearly.

Not: accept everything, queue unboundedly, let latency climb until every request times out, exhaust memory, crash, restart to a cold cache, be immediately overwhelmed again, and remain down for an hour after the spike ended. That is the default behaviour, and it converts a 20-minute traffic event into a 90-minute outage.

The mechanisms

1. Admission control at the earliest point. Reject before doing expensive work. A request rejected after the database query has already cost you the query — and under overload, that cost is exactly what you cannot afford. Rejecting at the edge is cheapest.

2. Concurrency limits, not just rate limits. What saturates the service is in-flight work. Adaptive limits that adjust to observed latency track real capacity better than a static number.

3. Bounded queues with an explicit full policy. An unbounded queue converts a throughput problem into a memory problem and then into an outage, and it makes recovery time unbounded.

4. Prioritisation. Shed low-value traffic first, using a classification decided before the incident and carried as a request attribute: a user mid-transaction outranks a new arrival; a non-retryable write outranks a refreshable read; a paying tenant may outrank a free one if that is the agreed policy. This cannot be invented under pressure.

5. Fast, clear rejection. HTTP 429 or 503 with Retry-After. This is not a courtesy — it is what stops a well-behaved client retrying immediately and making it worse. Silent drops guarantee retries.

6. Load shedding before the knee, not after. The knee — where latency starts rising non-linearly — is the real capacity limit, and it is well below the point where errors appear. Shedding should begin there.

7. Backpressure to upstreams, so callers slow down rather than piling on.

Recovery, which is half the requirement

When traffic returns to normal the service must recover without a restart. That requires:

• Bounded queues, so there is no enormous backlog to drain.
• No unbounded memory growth during the overload.
• Circuit breakers that close cleanly rather than sending a thundering herd at a recovering dependency.

Recovery behaviour is tested even less often than failure behaviour and causes just as many long outages.

The comparison worth stating

Two systems with identical capacity behave completely differently at 150%: one sheds and recovers instantly; the other collapses and stays down. The capacity number is the same. The failure mode is the architecture, and it is what stress testing exists to reveal.