# API Gateway Platform — Architecture One-Pager

*API Gateway Platform · Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-09 · 21 views · 18 architecture decision records*

**The request plane never calls the configuration plane.**

A SaaS product that other people build on has one component every external call passes through, and it is the component most able to take the whole product down. It must authenticate half a million credentials it did not issue, enforce four layers of quota without letting one tenant starve another, route to 800 published endpoints across three live API versions, let a hundred and twenty service teams ship and retire versions without a gateway change, and add so little latency and so little failure probability that nobody tries to route around it. The hard part is not any one of those capabilities. It is that each of them naturally wants to look something up — a key, a limit, a route, a policy — and a lookup on the request path is a dependency whose outage is the product's outage.

Three planes. A request plane of stateless Envoy proxies on GKE, behind a global anycast load balancer, which makes every per-request decision from configuration, credentials and counters already resident in the process. A control plane — config API, credential store, product catalogue, rollout controller, audit — that publishes immutable signed configuration versions the proxies pull, and is never called during a request. And a thin band of shared runtime state, of which the rate-limit counter store is the single deliberate exception to the rule, and is therefore designed to be degradable rather than highly available. Evidence flows upward and asynchronously: access records, metrics and audit events, out through Pub/Sub into BigQuery and an immutable ledger, never blocking a response.

## What it is, and what it is not

- **A policy boundary with a routing table** — not a routing table with some policy attached, which is what a gateway becomes when routing is designed first.
- **The trust boundary of the product** — not one hop among several, with upstreams re-authenticating because they do not quite believe it.
- **A place where a dimension is given up under failure** — not a place where availability is given up under failure.
- **The owner of the published contract** — not a transparent proxy whose behaviour is whatever the upstream happens to do this week.

## The decisions that are the architecture

1. **The request plane never calls the configuration plane** (ADR-01) — Route, credential, scope, limit, version and variant are all read from state resident in the proxy, loaded asynchronously from a signed configuration version. The control plane is never on the hot path, which is what licenses its lower availability target and turns its outage into "no new rollouts" rather than "no API".
2. **Configuration is an immutable artefact, pulled** (ADR-02) — A change becomes a signed, versioned snapshot before it is distributed, and proxies pull it. Revert is selecting an earlier version. Distribution survives an arbitrarily long control-plane outage because nothing has to be pushed at the moment it is needed.
3. **Revocation is a push with a measured coverage figure** (ADR-03) — ADR-01 forbids a synchronous revocation check, so revocation becomes a propagation SLO rather than a consistency guarantee — and an SLO needs evidence, so every proxy acknowledges its denylist version and coverage is a number someone is paged about.
4. **Fail open on reads, fail closed on writes** (ADR-04) — When identity is unreachable past the cache TTL, read routes serve on stale verifiers and write routes refuse. Splitting the posture by consequence rather than answering it once for the platform is the only version of this decision that survives contact with a real incident.
5. **Limits admit locally and reconcile centrally** (ADR-05) — Local buckets admit optimistically within a per-instance share and reconcile against a shared counter, buying a bounded ≤ 2% overshoot inside a 5 ms budget. When the counter store goes, ceilings tighten rather than disappear.
6. **Billing never reads a counter** (ADR-06) — Counters are approximate by design and are not durable. Billable usage is derived from access records, which are. Conflating the two is how an availability decision quietly becomes a revenue decision.
7. **The gateway signs, the upstreams trust nothing else** (ADR-08) — Client identity headers are stripped at admission and replaced with a signed context over mTLS. Raw credentials never cross into the service zone, so an upstream compromise yields no callable credential.
8. **Sunset is gated on evidence, not on a date** (ADR-10) — A version is retired when the platform can show zero active callers, or when someone records an override with their name on it. Dated per-tenant pins absorb the two customers who will not move, instead of the date slipping for everyone.
9. **No request or response bodies are logged** (ADR-18) — A gateway that logs payloads becomes the largest concentration of customer data in the company, inside the component with the widest reach. Logging is per-field opt-in, so a new header is not captured by default.

## Why this should still be right in ten years

An API gateway outlives the services behind it, because the contract it publishes is the thing customers wrote code against. These are the properties that should survive a change of proxy, a change of cloud, and the next credential format the industry agrees on.

- **The headline rule names no technology.** "The request plane never calls the configuration plane" says nothing about Envoy, GKE, Spanner or Google Cloud. It is a statement about which dependencies may sit on a request, and it stays true through any replacement underneath it. It is also the property a future team would most easily discard — by adding one convenient lookup for one urgent feature — which is why the resident-state boundary is drawn on the component view and the config pull is the only channel into it.
- **Degrade a dimension, not the service.** Every dependency on view 13 gives up accuracy, freshness, fidelity or the ability to change. Nothing gives up availability. That rule is a design discipline rather than a mechanism, and it applies equally to a dependency that does not exist yet: when someone proposes a new hot-path call, the question is which dimension its outage will cost.
- **The published contract is an asset with a lifecycle.** Versions, deprecation signalling, evidence-gated sunset and dated pins exist because external code cannot be refactored on demand. That is a property of having an ecosystem, not of HTTP or of REST; it transfers unchanged to whatever protocol the next decade prefers.
- **Credential design follows revocation, not the other way round.** Token lifetime is chosen against the revocation SLO — the two are the same decision wearing different names. Teams reliably discover this the other way round, by picking a convenient lifetime and then finding they cannot recall a leaked credential. Stating the SLO first survives any change of credential format.
- **Every rejection is debuggable by a stranger.** The people this platform refuses are mostly people the company has never met, working from documentation. An error that does not say which of four limits was hit, or which scope was required, is an error that becomes a support ticket. That is a property of having external users and it does not decay.

## Non-functional targets

The qualities that decide whether this design is the right one, each with the mechanism that delivers it and the view where it is drawn. Targets are stated assumptions from ask.md.

| Quality | Target | How it is met | View |
|---|---|---|---|
| Gateway-added latency | p50 ≤ 3 ms, p95 ≤ 8 ms, p99 ≤ 15 ms in-region | Every policy decision reads resident state; exactly two network hops leave the pod, and the counter call carries a 5 ms timeout with no retry. | 12 |
| Data-plane availability | ≥ 99.99% per region, ≥ 99.995% globally | Two active regions behind one anycast VIP, each sized for the whole load at burst; three zones per region with one zone of headroom. | 16 |
| Independence from the control plane | Zero data-plane impact from a control-plane outage | Proxies serve indefinitely on the resident configuration version and resume polling on recovery without a restart. | 13 |
| Credential revocation | Effective globally within 30 s | Pushed denylist with per-proxy version acknowledgement; coverage below 100% at 60 s pages someone. | 15 |
| Configuration propagation | ≥ 99% of instances within 10 s, 100% within 60 s | Signed snapshots pulled from object storage, version reported per pod, split-brain past 60 s alerted. | 09 |
| Quota accuracy | ≤ 2% overshoot at p99; hard ceilings exact within 1 s | Local admission against a per-instance share, reconciled with a sharded shared counter. | 14 |
| Tenant isolation | No tenant can cause another's requests to fail | Counter shards keyed by principal, per-route concurrency caps, tier-aware shedding, per-upstream circuit breaking. | 13 |
| Change safety | Auto-revert ≤ 90 s of breach; operator revert ≤ 30 s | Canary pods then canary traffic, scored on a rolling window; N−1 retained as a published snapshot. | 17 |
| Fault attribution | Gateway faults separable from upstream faults in every view | Fault attribution is its own signal family, tagged at emission rather than derived later. | 18 |
| Regional evacuation | Client-visible RTO ≤ 120 s | Health-check-driven anycast withdrawal into a region already carrying live traffic at full capacity. | 16 |

## Scope

**In scope**

- TLS termination, admission protections and north–south routing for every published API hostname, with the gateway as the only ingress path.
- Authentication for API keys, OAuth bearer tokens, partner JWT assertions and mTLS, with scope authorisation enforced per route.
- Four-scope quota and rate limiting — credential, application, tenant, route — with plan-tier defaults, dated per-tenant overrides and tier-aware shedding.
- API products with versioned lifecycle: preview, GA, deprecated, sunset; deprecation signalling, active-caller reporting and evidence-gated sunset.
- Progressive delivery at the edge: weighted splitting with sticky assignment, shadow traffic, health-gated automatic rollback, and configuration itself rolled out the same way.
- A self-service developer portal: registration, applications, credential issue and rotation, sandbox credentials, and the same usage numbers the operator sees.
- Structured access records, RED metrics, trace-context propagation, and an immutable audit ledger for authentication and configuration decisions.

**Explicitly out of scope**

- East–west service-to-service traffic, which belongs to the service mesh.
- Upstream business logic, schema design and persistence.
- Asynchronous outbound delivery to customer endpoints, which belongs to the webhook delivery platform.
- WebSocket and gRPC-streaming ingress, deferred to Phase 3.
- Gateway-side translation between API versions, deferred with its reasoning recorded in ADR-12.
- Cellular isolation per tenant tier and per-tenant data residency routing, deferred to Phase 3 with the trigger named in ADR-17.

## What a four-week prototype should prove

Four of this architecture's claims are cheap to test and expensive to be wrong about. A prototype that proves these can be built on; one that skips them is a demonstration of proxying HTTP, which is the easy half.

1. The latency claim with a real credential mix: drive representative traffic at a realistic cache miss rate and measure gateway-added p99 with the counter call in place. If the miss rate pushes p99 past 25 ms, the token-lifetime decision in ADR-03 has to be reopened before anything else is built.
2. The revocation SLO end to end: revoke a credential under load and measure the time until every proxy in both regions refuses it, including the coverage acknowledgement path. A 30 s SLO nobody has measured is a number in a document.
3. Counter overshoot under a hot tenant: run a single principal hard enough to concentrate on one shard and measure the actual overshoot against the ≤ 2% claim. This is the number most likely to be worse than assumed, and it decides whether local admission is viable at all.
4. A deliberately bad configuration version: push a routing change that breaks one route, and confirm the canary detects it, halts, and reverts within 90 s — and that a low-traffic route returns "insufficient evidence" rather than "healthy" (ADR-13).

- Control plane down: stop the Config API, the credential store and the snapshot publisher for four hours and confirm every proxy keeps serving on its resident version, that a pod restarted during the outage comes up serving, and that polling resumes on recovery without a restart.
- Counter store pulled: kill the Memorystore shard under load and confirm limits tighten to the per-instance ceiling rather than disappearing, that the 5 ms timeout does not become latency, and that billing is unaffected because it never read the counters (ADR-06).
- Identity unreachable past the TTL: block the credential store and confirm read routes keep serving on cached verifiers while write routes refuse with a distinguishable 503, and that a credential already on the denylist is still refused throughout (ADR-04).
- Quiet-route canary: roll out a deliberately broken change to a route taking two requests a minute and confirm the rollout holds on "insufficient evidence" rather than promoting to 100% (ADR-13).
- Revocation under load: revoke a credential in production-like traffic and measure time to global refusal plus the coverage acknowledgement, confirming the operator is told "propagating" and not "revoked" (ADR-03).
- Hot tenant: drive a single principal hard enough to concentrate on one counter shard and confirm no other tenant's p99 moves — the isolation claim stated as a measurement rather than a design intent.
- Telemetry saturation: throttle the usage pipeline and confirm the platform samples down, then drops, without any change to request latency, and that authentication and configuration events survive the whole episode (ADR-16).

## Open risks, carried rather than hidden

| Risk | If it lands | Response |
|---|---|---|
| The credential cache miss rate is higher than assumed | The tail latency of the whole platform stops being a property of the proxy and becomes a property of the credential store, which is exactly the dependency ADR-01 was drawn to keep off the request path. A 15 ms p99 becomes unachievable without weakening the revocation SLO. | Measure it in the prototype before committing to token lifetimes. The lever is ADR-03: shorter opaque-credential caching improves revocation and worsens tail latency; the two must be traded explicitly rather than discovered. |
| A single global fleet is a single global blast radius | One bad configuration version, one filter regression, or one hot tenant saturating shared capacity affects every customer at once. Every mitigation here — canary, tier shedding, counter sharding — narrows the window rather than the population. | Cells per plan tier are deferred rather than dismissed, with the trigger stated in ADR-17: the first incident whose blast radius would have been contained by cells, or the first enterprise contract requiring isolation. The cost of cells is known in advance so the decision is not made under pressure. |
| The canary cannot see a regression on a low-traffic route | Progressive delivery reports healthy because nothing measurable happened, and the regression ships at 100%. This is a silent failure of the safety mechanism itself, which is worse than not having it. | The rollout controller must distinguish "healthy" from "insufficient evidence" and hold or escalate on the latter (ADR-13). Low-traffic routes may need a longer window, synthetic traffic, or an explicit human gate — the design records that the problem is unresolved rather than averaging it away. |
| The fail-open read posture is exploited during an identity outage | A credential revoked just before the outage keeps working on read routes for the duration, and the window is unbounded because it is the outage's length, not a TTL. | The denylist is pushed and survives the identity outage, so the common case — a known-leaked credential — is still refused. The residual exposure is a revocation issued during the outage, which is accepted explicitly in ADR-04 and bounded by making write routes fail closed. |

The reasoning behind every component and technology choice is in the [Architecture Decision Record](decision-record): 18 records across 6 areas, each with the alternatives that lost and what the choice costs.