Reserve the pessimistic figure — estimated input plus maximum output — then give back what was not used. Reserving the optimistic figure would let a tenant with large max_output values quietly exceed TPM by a wide margin.
The reaper is what makes pessimism affordable. A client that vanishes after ALLOW costs its tenant at most 120 s of held quota, not the rest of the window, so the conservative reservation never becomes a denial-of-service against the tenant itself.
Concurrency slots follow the identical lifecycle. A slot is held from ALLOW and released on commit, failure or sweep, which is precisely FR4.
Numbers
Worked example from the brief: estimate 5,000 in + 2,000 out = 7,000 reserved; actual 5,800; 1,200 released.
Reservation TTL 120 s, held in a Valkey sorted set scored by expiry so the reaper is a single ZRANGEBYSCORE per shard per second.
Measured over-reservation across the fleet: 14% of tokens reserved are released unused, which is the cost of not knowing output length in advance.
Risks
A tenant that sets max_output far above what its prompts ever produce systematically under-uses its own TPM. The console surfaces the release ratio per tenant so this is visible and correctable.
Streaming responses only reveal the output count at stream end. A stream abandoned mid-flight commits the tokens observed so far, which under-charges slightly and is accepted.
The reaper is a single logical writer per shard. If it stalls, held quota accumulates; its own liveness is alerted separately from limiterd's.
