Your service is at capacity and must reject some traffic. What do you shed, and how do you decide?

What the interviewer is testing

Whether you treat shedding as a business prioritisation decision rather than a technical default.

Why shedding at all

The alternative is not "serve everything". It is congestive collapse: queues grow, latency exceeds usefulness, callers time out and retry, and the system delivers nothing while working flat out. Shedding is how a system stays useful at overload instead of becoming uniformly useless.

What to shed, in priority order

Requests whose deadline has already expired. Free capacity nobody is waiting for — check on dequeue and drop. This is the cheapest and most obviously correct category.

Low-value operations by business priority. Recommendations, analytics events, non-essential enrichment, prefetching. Payment completion, authentication and core reads survive. This ranking is a business decision and must be agreed in advance, not invented during the incident.

Expensive queries — a small number of costly requests can consume the capacity of many cheap ones.

By tenant or caller, enforcing fairness so one client cannot consume the pool, and protecting higher-tier customers where the commercial model supports it.

Retries before first attempts, since a retry has already had one chance and shedding it does not deny anyone a first response.

Randomly, as the crude fallback when nothing better is available.

The mechanics

Reject early and cheaply, at the edge or on admission — a rejection costing as much as serving has not helped.

Return a clear signal: 429 or 503 with a retry hint, so well-behaved clients back off rather than retrying immediately.

Trigger on the right measurement: concurrency in flight and queue depth, not CPU. Queue depth predicts collapse; utilisation lags it.

What a strong answer adds

Testing it. A shedding mechanism that has never fired under real conditions usually has a bug — an incorrect priority classification, a rejection path that is itself expensive, or a threshold that never triggers. A load test that drives the system past capacity is the only way to know.

And instrumenting what was shed and why, so the business can see the cost of the capacity decision.

Common weak answers

Random shedding as the design. Autoscaling as the answer, which takes minutes and may be constrained by quota exactly when needed.