A rate limiter protects an API from abuse, but enterprise customers have legitimate predictable bursts. How should quotas differ by tenant, endpoint, priority and available capacity?

Why a single global limit is wrong

It has to be set low enough to protect the system from the worst caller, which makes it too low for the legitimate large one. The result is that the customers who matter most hit the limit most often, and the limit gets raised for them individually — quietly deleting the control for exactly the accounts capable of causing the most damage.

The dimensions that a workable scheme uses

• Per tenant, so one caller cannot consume shared capacity. The fundamental unit.
• Per endpoint or cost class, because a lightweight read and an expensive write should not share a budget. Weighting by cost rather than counting requests is what makes this meaningful.
• Burst allowance separate from sustained rate. A token bucket with a large bucket and a modest refill rate accommodates a legitimate burst — a batch reconciliation run, a nightly sync — while bounding sustained consumption. This single mechanism resolves most enterprise complaints about rate limits.
• Priority classes, so that when capacity is genuinely scarce, a payment initiation is served and a reporting query is shed.
• A global backstop that reduces all limits proportionally when the system is under real pressure, because per-tenant limits summed across all tenants can still exceed capacity.

What the API must communicate

• Limit, remaining and reset in every response, so a well-behaved client can pace itself. A client that cannot see its budget will discover it by failing.
• 429 with Retry-After, and clients told to honour it. Without this, rejection triggers immediate retry and the limiter amplifies the load it exists to reduce.
• Distinguish "you exceeded your quota" from "the system is shedding load", since the correct client behaviour differs: back off and reduce for the first, retry shortly for the second.

The implementation detail that breaks things

Limits enforced per instance rather than globally. With twenty gateway instances and a per-instance limit, the effective limit is twenty times the intended one, and it varies with fleet size and load balancing. A shared counter is required, and its own availability then matters — which is why the usual design is a shared store with local approximation and a fail-open posture, accepting brief over-admission rather than turning the rate limiter into a single point of failure.