AI Gateway
A shared proxy in front of model providers that centralises routing, keys, quotas, caching, logging and safety policy.
Once more than one application calls a model, the same problems appear in each: provider credentials distributed everywhere, no view of spend, no per-team quota, no way to fail over when a provider degrades, no consistent logging of prompts and responses, and no central place to apply a safety policy.
An AI gateway is the API gateway pattern applied to model traffic. It gives model-agnostic routing, semantic or exact response caching, token accounting per team and per feature, retry and failover across providers, and one place where prompt-injection filtering and output moderation are enforced.
It is also where the switching cost is contained. Applications that call the gateway rather than a provider SDK can change model or provider as a configuration change.