A payments API must stay available on the highest-traffic commerce day of the year. How should rate limiting and load shedding be structured so that critical traffic survives while non-critical traffic is sacrificed?

The distinction that organises the design

Rate limiting is about fairness. Load shedding is about survival. They are different mechanisms with different triggers, and conflating them produces a system that either rejects legitimate traffic in normal conditions or fails to protect itself in abnormal ones.

• Rate limits are per-tenant, apply always, and exist so no single caller can monopolise shared capacity. They are predictable and documented, and a well-behaved integrator never encounters them.
• Load shedders activate only under stress, are global rather than per-tenant, and drop traffic by priority class to keep the system serving the things that matter.

Classifying traffic

The classification must be explicit, assigned per endpoint, and agreed with the business — not inferred at runtime:

• Critical: creating a charge, capturing a payment, authorising a transaction. Failing these loses money and trust irrecoverably, and they are typically a minority of request volume.
• Important: reading a payment's status, webhook delivery.
• Deferrable: reporting queries, list endpoints, historical exports, dashboard analytics. These often dominate request volume and can be shed with almost no user harm, which is what makes the whole scheme work.

The critical insight is that the volume and the value are inversely correlated. Shedding the bulk of traffic costs little; that is the asymmetry the design exploits.

Separate shedders

Running an independent load shedder per class rather than one global one matters because it prevents a flood of low-priority traffic from ever competing with critical traffic for the shedding decision itself. Each class has its own capacity allocation and its own threshold, so the reporting endpoints being overwhelmed cannot consume the budget that charge creation needs.

Additionally, a per-worker shedder that rejects based on local queue depth provides a fast, dependency-free safety valve that works even when the coordination layer is degraded.

The mechanics that matter

• Token bucket per tenant for rate limits, allowing bursts while bounding the sustained rate — bursts are normal and legitimate, and a fixed-window limiter rejects them incorrectly.
• Reject fast and cheaply. A rejection must cost far less than serving; if the limiter itself is expensive, it is the bottleneck under attack.
• Retry-After and clear headers — remaining quota, reset time — so well-behaved clients back off correctly rather than retrying immediately and amplifying.
• Queue depth and latency as shedding triggers, not just request rate. Rate is a poor proxy for stress because the cost per request varies enormously.
• Reject at the edge, before the request consumes backend resources.

Preparing for the known peak

Peak commerce days are scheduled, which changes everything: pre-scale rather than autoscale, since autoscaling reacts after the spike has already caused damage. Load test at the projected peak weeks in advance. Freeze changes. Pre-agree the degradation ladder with the business — which features go first, who decides, and what customers are told — because that conversation is impossible to have well at the moment it is needed.

And rehearse the shedding: activating a load shedder for the first time during the busiest hour of the year is how a protection mechanism becomes an incident.