Priority Load Shedding

When a system is overloaded, it will drop work. The only question is whether it chooses which work. A system without prioritisation drops whatever happens to arrive when the queue is full, which means the most valuable request in the system has the same survival probability as the least valuable one.

Priority load shedding assigns every request a criticality class and, under stress, sheds from the bottom up — preserving the operations whose failure is expensive while sacrificing the ones whose failure is merely inconvenient.

Why it matters

Volume and value are usually inversely correlated, and that asymmetry is what makes the technique so effective. In a commerce system, payment authorisation is a small fraction of requests and nearly all of the value; dashboards, list endpoints, reporting queries and analytics dominate the volume and can be shed with little harm.

So shedding 60% of requests may cost 3% of business value. No other reliability technique offers that ratio, and it requires no additional capacity — only the decision, made in advance.

The corollary matters too: a system without prioritisation is implicitly choosing to protect its cheapest traffic, because high-volume endpoints are more likely to be the ones occupying the queue when it fills.

Implementation patterns

• Criticality declared per endpoint or per operation, in code or configuration, agreed with the business rather than assigned by engineers — the classification is a commercial judgement wearing technical clothing.
• The class propagated through the whole call chain, so downstream services shed consistently. A request that is critical at the edge must not become anonymous three hops in.
• Independent shedders per class, so low-priority traffic cannot consume the capacity — or even the admission-decision capacity — that critical traffic needs.
• Local, per-worker shedding on queue depth, providing a fast, dependency-free valve that works even when the coordination layer is degraded.
• Trigger on saturation signals — queue depth, latency, concurrency — not on request rate, since per-request cost varies enormously and rate is a poor proxy for stress.
• Shed at the edge, before the request consumes backend resources; shedding after the expensive work is wasted effort with an error attached.
• Clear client signalling — 503 with Retry-After — so shed clients back off instead of amplifying.
• Rehearsal: activate shedders deliberately in a game day. The first activation should never be during the event it was built for.

Industry example

Payment platforms operating through peak commerce periods are the canonical case, and Stripe has described running separate load shedders for critical and non-critical traffic rather than one global mechanism — precisely so that a flood of reporting or dashboard traffic cannot compete with charge creation.

The same structure appears in every system with a scheduled, predictable peak: retail on sale days, streaming during live events, brokerages during market open. The common preparation is not extra capacity alone but a pre-agreed list of what gets turned off, with owners and trigger points, because that conversation cannot be held well while the incident is happening.

Failure scenarios

• No classification, so shedding is random and the valuable requests die with the rest.
• Everything classified as critical, which is the usual outcome when engineers assign the classes without business input, and which makes the mechanism inert.
• Class lost at a service boundary, so downstream tiers shed blind.
• Shedding triggered on request rate, missing overload caused by expensive requests at low rates.
• Expensive shedding, where the rejection path itself consumes significant resources.
• Shedding after the work is done, wasting the capacity that was supposed to be protected.
• Shed responses that clients retry immediately, converting protection into amplification.
• Never tested, so the mechanism has a bug that is discovered at the worst possible moment.

Trade-offs

Priority shedding means deliberately failing some users' requests while others succeed, which is a fairness decision with commercial and sometimes regulatory implications — a customer whose reports are shed while another customer's payments succeed may reasonably object, and the tiering must be defensible.

It also adds a classification burden to every new endpoint and a propagation requirement to every service boundary, both of which decay without enforcement.

And it is not a substitute for capacity. A system that sheds routinely is under-provisioned, and prioritised shedding can mask that by keeping the important paths green while the experience degrades broadly.

The trade is fairness and classification overhead in exchange for keeping the valuable operations alive during overload. For anything transactional or revenue-bearing, that is an obviously correct trade; for a system where all requests are of similar value, the machinery buys much less and simple fair queueing may serve better.

Interview question

"Traffic is 5× forecast and we can serve 40% of it. Tell me how the system decides what to drop, who made that decision and when, and what happens to the class label when the request crosses into a service that was written before we had classes."