Adaptive Admission Level

Load shedding is normally local: a service detects overload and rejects requests. That protects the service and wastes everything already spent on the request.

In a deep call graph the waste is the dominant term. A request rejected at the fifth hop has consumed capacity at four earlier services, and under overload that consumed-then-discarded work is a principal cause of the overload rather than a side effect of it.

An adaptive admission level fixes the direction of information flow. Each service computes, from its own load, the priority threshold at which it is currently admitting work, and returns that threshold to its callers. Callers stop sending anything below it. The shed then happens near the edge, at a cost of one unit of work instead of five.

Why it matters

It converts shedding from a local defence into a fleet-level control loop. Without propagation, every service in the graph independently discovers overload and independently rejects, and the system spends its remaining capacity generating work it will throw away.

It also makes the shed coherent. When each hop sheds independently, a user's request may succeed at three hops and fail at the fourth, producing a half-completed journey. A shared priority scheme means the decision is effectively made once, at the edge.

Implementation patterns

• Derive load from queuing time, not CPU. Average waiting time in the pending queue is a direct measure of backlog, is comparable across hardware and workload types, and moves before utilisation shows anything.
• Use two priority dimensions. Business priority alone drops whole features for everyone; user priority alone admits a random fraction of every request, so a user who succeeds at step one fails at step three. Combined, the least important features go first and the same users are consistently admitted, so those who get through finish.
• Propagate the threshold on the response path, as a header or a field, so no extra round trip is needed and the information is as fresh as the traffic.
• Carry the request's priority on every hop, including asynchronous ones, or the scheme has a hole exactly where the deep graphs are.
• Damp the control loop. Thresholds recomputed too often oscillate; a short averaging window with hysteresis is the usual answer.
• Make the edge authoritative on priority assignment, so an internal service cannot promote its own traffic.

Industry example

DAGOR, published by Tencent at SoCC 2018, is the documented instance: the WeChat backend profiles server load from the average waiting time of requests in the pending queue, admits by business and user priority, and propagates admission levels between services so that shedding is collaborative rather than local. The paper reports it in use across the WeChat backend for five years at that point, and notes that session-oriented admission tends to degrade user experience while user-oriented admission is as effective for load control.

Failure scenarios

• Oscillation, where too short an evaluation window makes the threshold swing and traffic alternates between flooding and starvation.
• Priority inflation, where every team declares its traffic critical and the scheme degrades to no scheme.
• A hole in propagation — an asynchronous hop, a third-party call, a cache-fill path — through which unpriced work re-enters the graph.
• Retry storms below the threshold, where rejected callers retry immediately and generate the load the shed was meant to remove.
• Correct shedding of the wrong thing, when the priority mapping was written once and never revisited against what the business actually values.

Trade-offs

When not to use it

When call graphs are shallow, the waste it removes barely exists. A two-hop path rejecting at the second hop wastes one hop's work, and a fleet-wide priority protocol is a large amount of machinery for that.

Most organisations should start with the cheaper 80%: per-dependency concurrency limits plus a deadline propagated from the edge. Concurrency limits stop a caller queueing unbounded work at a struggling dependency, and a propagated deadline lets each hop refuse work that cannot finish in time — which captures most of the "do not perform work that will be discarded" benefit without a priority taxonomy or a cross-team negotiation.

Interview question

Q: Your estate sheds load correctly at every service and still collapses under a traffic spike. Success rate falls further than the overload should explain. What is happening, and what would you change?

What a strong answer covers: that independent local shedding wastes all upstream work on every rejected request, and under overload that waste is the dominant load; propagating an admission threshold back to callers so the shed happens at the edge; deriving load from queuing time rather than CPU; the two priority dimensions and why user-consistent admission preserves complete journeys; retry behaviour below the threshold as the thing that undoes it; and the cheaper starting point of concurrency limits plus propagated deadlines.

Quick check

Quiz: Why does propagating an admission threshold beat shedding locally? Because a request rejected deep in the call graph has already consumed every earlier service's capacity, and under overload that discarded work is itself a principal cause of the overload.

Flashcard: What signal should drive an admission threshold, and why not CPU? Average waiting time in the pending queue: it measures backlog directly, moves before utilisation does, and means the same thing across different hardware and across CPU-bound and I/O-bound services.