Provider quotas are enforced globally, above and independent of tenant quotas. Every tenant can be within its own limit while the fleet collectively exceeds what the provider will accept — that is the case FR7 exists for.
Failover is opt-in per tenant policy. A tenant pinned to one provider for compliance or residency reasons receives 503 rather than a silent reroute to a different jurisdiction, and the policy records that choice explicitly.
Upstream 429s feed an adaptive throttle. The provider's own backpressure is treated as the most accurate signal available about its real limit, which is often lower than the documented one.
Numbers
Breaker trips on 3 failures in 2 s, half-opens after 10 s, and requires 5 consecutive successes to close.
Failover completes within 5 s of the trip, measured from first upstream error to traffic on the fallback route.
The self-hosted vLLM pool (Mixtral 8x7B on 16 A100s) is the last-resort route and is sized for 15% of peak, which is the deliberate cap on how much failover it can absorb.
Risks
Failover changes model behaviour. A prompt tuned for one model may produce materially different output on another; tenants opt in per model family and the ledger records which provider actually served each request.
A correlated outage across two managed providers pushes far more than 15% of peak at the vLLM pool. The pool sheds by tier rather than degrading for everyone.
Residency filtering depends on correct provider region metadata. It is treated as a controlled configuration item with a change review, not an ordinary policy edit.
