# API Gateway Platform — Architecture Decision Record

*API Gateway Platform · Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-09 · 21 views · 18 architecture decision records*

The argument these decisions serve is summarised in the [Architecture One-Pager](architecture-one-pager).

Eighteen decisions make up this architecture. Everything else across the twenty-one views is convention, sizing or consequence. Each record states the forcing question, the context that makes it hard, what was decided, how it is realised on Google Cloud, the options weighed, what the choice buys and costs, the conditions that would flip it, why it should still be right in ten years, and the lesson that transfers to a system that is not this one.

> **Status of this document.** This is a design, not a report on a running system. The rates, latencies, volumes, retentions and thresholds are the requirement's stated assumptions for a multi-tenant SaaS with a public developer ecosystem — 50,000 developer organisations, 500,000 live credentials, 800 routes across 120 upstream services, ≈ 10 billion requests/day — invented to be defensible and arguable rather than absent, and to be replaced by measured traffic before build. Four of them are the first numbers real data should overturn: the 15 ms p99 gateway overhead, which is what makes the resident-state design necessary rather than merely tidy; the 30 s revocation propagation SLO, which sets token lifetime and therefore the whole credential design; the ≤ 2% counter overshoot, which decides whether local admission is acceptable at all; and the credential cache miss rate, which is what actually sets the tail latency.

## How to read a record

- **Question:** The forcing question: why a decision was needed at all.
- **Context:** The requirement, the scale and the constraint that make it hard.
- **Decision:** What this architecture does, stated so it can be checked.
- **How it is realised on Google Cloud:** The concrete mechanism: which service or package, configured how, in which project.
- **Options weighed:** Chosen, rejected, deferred, or right elsewhere, with the reason for each.
- **Consequences:** What the choice buys and what it costs, both kept visible.
- **Choose differently when:** The conditions that would flip the decision for your system.
- **Why it holds up over time:** What keeps the decision right as scale, staff and technology change.
- **Lesson:** The principle that transfers beyond this platform.

## Decision map

**The boundary**: What a request is allowed to depend on, and what follows from making that list as short as possible.

- ADR-01 · The request plane never calls the configuration plane
- ADR-02 · Configuration is an immutable versioned artefact, distributed by pull
- ADR-08 · The gateway signs an identity context; upstreams trust that and nothing else

**Identity at the edge**: How half a million credentials the platform did not issue are verified in single-digit milliseconds, and what happens when the thing that knows about them is unreachable.

- ADR-03 · Opaque credentials cached as verifiers, signed tokens verified statelessly, both under one denylist
- ADR-04 · Fail open on read routes, fail closed on write routes

**Fairness and limits**: Keeping one tenant's traffic from becoming another tenant's outage, without a distributed transaction on every request.

- ADR-05 · Local admission against a per-instance share, reconciled with a sharded shared counter
- ADR-06 · Billable usage is derived from access records, never from counters
- ADR-07 · Overload shedding is by plan tier, and every shed request is recorded

**The published contract**: Versions, deprecation and retirement, for code the platform cannot refactor because other people wrote it.

- ADR-09 · Version in the path, one version per route, no gateway translation in v1
- ADR-10 · Sunset is gated on measured active callers, with dated per-tenant pins
- ADR-12 · Gateway-side version translation is deferred, and the reason is ownership

**Change and delivery**: How a change reaches a fleet that is serving, and how it is taken back when it is wrong.

- ADR-11 · Proxy filters are code, and ship through the same canary as configuration
- ADR-13 · The rollout controller must be able to return insufficient evidence

**Running it**: Topology, scaling signals, evidence and the data the platform refuses to hold.

- ADR-14 · Two regions active–active behind one anycast VIP, each sized for the whole load
- ADR-15 · Autoscale on connection count and CPU saturation, not request rate
- ADR-16 · Telemetry is sampled; policy decisions never are
- ADR-17 · One shared fleet in v1; cells are deferred with the trigger named
- ADR-18 · No request or response bodies are logged; field capture is opt-in

## Technology by capability

Every capability mapped to what realises it on Google Cloud, what the obvious alternative was, and why the choice fell where it did. Google Cloud was chosen for stack rotation across this practice rather than because the topic belongs to it: an API gateway is the most cloud-portable component in a SaaS estate, and every requirement in ask.md survives a change of provider.

| Capability | Choice | Origin | Credible alternative | Why this one | Record |
|---|---|---|---|---|---|
| Global ingress and TLS termination | Global external Application Load Balancer, one anycast VIP, Google-managed certificates | Google Cloud | Self-managed proxy fleet on regional addresses with DNS failover | A single anycast VIP makes regional evacuation a health-check outcome rather than a DNS propagation race, which is what allows a 120 s client-visible RTO. DNS failover cannot make that promise because resolver caches are outside the platform's control. | ADR-14 |
| Edge protection | Cloud Armor — managed WAF rules, DDoS absorption, geo and IP policy per API product | Google Cloud | WAF filters inside the proxy fleet | Volumetric absorption has to happen before traffic reaches capacity that is sized for legitimate load. A WAF inside the fleet protects the upstreams and not the fleet, which is the wrong half of the problem. | ADR-14 |
| Request-plane proxy | Envoy on GKE regional node pools, one pod per unit of capacity | Open source on Google Cloud | A fully managed API gateway product | The resident-state design in ADR-01 requires control over what the proxy caches and when it refreshes. A managed gateway owns those decisions, and they are precisely the ones this architecture is built from. | ADR-01 |
| Configuration store | Spanner, versioned rows with a digest and parent version | Google Cloud | Cloud SQL with read replicas | Configuration is small, globally read, and must be strongly consistent on write across regions so two operators cannot create conflicting versions. That is the one workload where a globally consistent store earns its cost. | ADR-02 |
| Configuration distribution | Signed snapshots in a dual-region Cloud Storage bucket, pulled by the proxies | Google Cloud | xDS streaming from a control-plane service | A pod restarting during a control-plane outage must still be able to come up serving. Object storage is a far smaller dependency than a control-plane service, and a signed snapshot is verifiable without trusting the channel. | ADR-02 |
| Rate-limit counters | Memorystore for Redis, sharded by principal, one Lua evaluation per request | Google Cloud | Spanner, or per-instance counters only | The counter is the one hot-path dependency and needs single-digit-millisecond reads with atomic multi-scope evaluation. A durable store would be over-specified for data that is explicitly safe to lose (ADR-06). | ADR-05 |
| Credential store | Spanner, verifiers and issuer keys only, no recoverable secrets | Google Cloud | Secret Manager per credential | The access pattern is a point read per credential at high rate from two regions, and the platform deliberately holds no secret to protect — only a verifier. Secret Manager solves a problem this design removes. | ADR-03 |
| Revocation distribution | Pub/Sub fan-out to every proxy, acknowledged with a denylist version | Google Cloud | Short cache TTLs and polling | A 30 s revocation SLO with a 5-minute cache TTL needs a push. Polling fast enough to meet the SLO would turn the credential store into a hot-path dependency by another name. | ADR-03 |
| Usage and evidence pipeline | Pub/Sub into BigQuery, partitioned by day and tenant | Google Cloud | Direct writes to an analytics store from the proxy | The emission path must absorb backpressure without touching the request path. A buffered bus is what makes "sample down, then drop" a telemetry decision rather than a latency event. | ADR-16 |
| Audit ledger | Object storage with retention lock, written from the same bus, 7-year retention | Google Cloud | The analytics store with a long partition expiry | Audit needs immutability that survives an operator with database permissions. A retention lock is a different guarantee from a long retention setting, and the difference is the whole point. | ADR-18 |
| Control-plane services | Cloud Run for the Config API, developer portal and rollout controller | Google Cloud | The same GKE clusters as the proxies | Running the control plane on the serving clusters would put it in the same failure domain as the thing whose outage it is supposed to survive. Separation here is the point, and the scale-to-zero economics are a free extra. | ADR-01 |
| Developer portal and product catalogue | Apigee-style product model — API products, plans, subscriptions — implemented over the Config API | Google Cloud pattern | A bespoke portal with routes as the unit of subscription | Subscribing to a product rather than a route is what makes plan tiers, scopes and version pins expressible at all. Route-level subscription cannot express "this customer is on v1 until March". | ADR-10 |
| Upstream authentication | Mutual TLS with workload identity, plus a signed request-bound identity assertion | Google Cloud + open standard | Trusted headers on a private VPC | Network reachability is not authentication in an estate of 120 services. Binding the assertion to the request identifier is what stops it being replayed onto a different call. | ADR-08 |
| Metrics, traces and alerting | Cloud Monitoring and Cloud Trace, with fault attribution as a first-class dimension | Google Cloud | Self-hosted Prometheus and a trace backend | The differentiating requirement is not the backend but the tagging: separating gateway faults from upstream faults at emission. That is a filter decision, and it works the same on any backend. | ADR-16 |

## The decisions, and the alternatives that lost

### The boundary

*What a request is allowed to depend on, and what follows from making that list as short as possible.*

#### ADR-01 · The request plane never calls the configuration plane

**Status:** Accepted  ·  **Shown on views:** 07, 12, 13

*When the gateway needs to know a route, a credential, a scope, a limit or a version, may it look that up while a request is waiting?*

**Context.** Every capability this platform has naturally wants a lookup. Authentication wants the credential store. Authorisation wants the grant table. Rate limiting wants the plan. Routing wants the route table. Each lookup is individually reasonable and individually fast, and together they make the product's availability the product of five dependencies' availabilities, with the gateway's latency set by whichever of them is slowest at the 99th percentile. The failure mode is not gradual: a control-plane deploy that adds 40 ms to a config read becomes a 40 ms regression on every API call in the company, and a control-plane outage becomes a total outage of a product whose services are all healthy.

**Decision.** Every per-request decision is made from state already resident in the proxy process: a signed configuration snapshot, a credential verifier cache, a pushed revocation denylist, and local counters. The control plane, the configuration store, the credential database and the product catalogue are never called during a request. They are refreshed asynchronously, in the background, on their own schedule.

**How it is realised on Google Cloud.** Envoy pods on GKE hold the configuration snapshot in memory, pulled from a signed object in a dual-region Cloud Storage bucket and refreshed on a poll. The credential cache and denylist are populated by a background refresher and a Pub/Sub subscription respectively. Spanner, the Config API on Cloud Run and the product catalogue have no ingress path from the request-serving path at all — enforced by network policy and by service-account scoping, not by convention.

| Option | Verdict | Reasoning |
|---|---|---|
| Resident state only; the control plane is refreshed asynchronously | Chosen | Makes the control plane's availability target legitimately lower than the data plane's, and puts a hard ceiling on gateway-added latency. Costs eventual consistency on every kind of change, which is why revocation becomes ADR-03 and propagation becomes a measured SLO. |
| Lookup with an aggressive cache | Rejected | This is what most gateways do and it works until the cache misses during the incident. A cache is a latency optimisation; it is not an availability boundary, because its cold path is still a dependency and cold paths correlate with outages. |
| A sidecar that owns the lookups | Rejected | Moves the network hop rather than removing it, and adds a second process whose failure modes are now on the request path. Attractive because it looks like isolation; it is really indirection. |

**What it buys**

- A control-plane outage stops change and changes nothing about traffic — which is what makes a 99.9% control-plane target defensible against a 99.99% data-plane target.
- Gateway-added p99 contains no control-plane round trip, so the latency budget is a property of the proxy rather than of the slowest management service.
- Every dependency failure on view 13 degrades a dimension instead of availability, because the dependencies were removed from the path before the failure analysis started.

**What it costs**

- Every change is eventually consistent, and each one needs a stated propagation SLO plus the machinery to measure it.
- Revocation cannot be immediate, which pushes real complexity into ADR-03 and leaves a stated exposure window.
- Proxy memory becomes a sizing constraint, because configuration and credential state are resident rather than remote.

**Choose differently when.** If the credential population grew past what can be held in each proxy — tens of millions of verifiers rather than hundreds of thousands — resident credential state would stop being viable and the design would have to move to signed, self-describing tokens with a small denylist, making ADR-03 the primary decision and this one its consequence.

**Why it holds up over time.** The rule names no technology. It is a statement about which dependencies may sit on a request, and it survives replacing the proxy, the cloud and the credential format. It is also the property a future team would most easily erode — one convenient lookup for one urgent feature — which is why it is drawn as a boundary on the component view rather than written down as a guideline.

> **Lesson.** Availability is not something a component has; it is something a component inherits from everything it calls while someone is waiting. The cheapest way to be available is to be calling nothing.

#### ADR-02 · Configuration is an immutable versioned artefact, distributed by pull

**Status:** Accepted  ·  **Shown on views:** 09, 17

*How does a routing or policy change reach a serving fleet, and what exactly is reverted when it is wrong?*

**Context.** ADR-01 means the fleet runs on cached state, and cached state that nobody can identify is unmanageable: "which proxies have the new limit?" has no answer, "revert" means re-running a pipeline and hoping, and two proxies can silently disagree about the same request. The alternative failure is equally common: a streaming push protocol that works beautifully until the control plane is down at the moment a pod restarts, and the pod comes up with no configuration at all — turning a management outage into a serving outage through the back door.

**Decision.** A change is committed as an immutable, monotonically versioned, signed configuration snapshot before it is distributed. Proxies pull the snapshot, verify the signature, and report the version they are actually running. A rollout is a decision about which version a set of proxies should be on; a revert is selecting an earlier version, which is already published and already verified.

**How it is realised on Google Cloud.** The Config API writes the change to Spanner as a new version row with a digest and a parent version. A publisher renders the snapshot, signs it, and writes it to a dual-region Cloud Storage bucket under its version. Proxies poll for the version they have been told to run, verify, and load. Version N−1 is always retained. Each pod reports its resident version on a heartbeat, so fleet coverage is a table rather than an assumption.

| Option | Verdict | Reasoning |
|---|---|---|
| Immutable signed snapshots, pulled by the data plane | Chosen | A pod restarting during a control-plane outage can still fetch a known-good snapshot from object storage, because object storage is the dependency, not the control plane. Costs a propagation delay measured in seconds. |
| Streaming push from the control plane (xDS-style) | Rejected | Faster and the industry default, but it makes the control plane a runtime dependency of every pod start, which is precisely the coupling ADR-01 removes. Right for a mesh, where the control plane is already in the failure domain. |
| Configuration baked into the container image | Rejected | Maximum immutability, and it makes a limit change a fleet redeploy. A platform where changing one tenant's quota takes a rolling restart will grow a side channel within a month, and the side channel will not be versioned. |

**What it buys**

- Revert is a selection, bounded by how fast a pod can load a snapshot, and the thing being reverted to is already known to parse.
- "What was this fleet running when that request failed?" is answerable, because access records carry the config version (view 11).
- Progressive rollout of configuration becomes possible at all, since a subset of pods can be told to run a different version.

**What it costs**

- A propagation window in which the fleet is split across two versions, which has to be bounded and alerted rather than wished away.
- Snapshot rendering and signing is a build step with its own failure modes, and a stale publisher silently freezes the platform's ability to change.
- Storage and retention of every version, and the discipline to never mutate one.

**Choose differently when.** If propagation latency had to drop to sub-second — an adaptive policy reacting to live traffic, say — pull would stop being viable and a streaming channel would be needed, accepting the control-plane coupling and compensating with a resident fallback snapshot for cold starts.

**Why it holds up over time.** Immutable, content-addressed, signed artefacts with an explicit version are how every durable delivery system ends up working, from package managers to container registries. The shape outlives the storage technology, and the discipline it enforces — that the deployed thing has a name — is what makes every later operational question answerable.

> **Lesson.** If you cannot name what is currently running, you cannot roll it back, cannot attribute a failure to it, and cannot tell whether two machines agree. A version number is not metadata; it is the precondition for every operational answer.

#### ADR-08 · The gateway signs an identity context; upstreams trust that and nothing else

**Status:** Accepted  ·  **Shown on views:** 12, 20, 21

*How does an upstream service know who is calling, and how much is it required to re-verify?*

**Context.** If upstreams re-authenticate the caller, the platform has centralised nothing: every service needs credential-verification code, every credential type change is a hundred-and-twenty-service migration, and the services will diverge in how strictly they enforce scopes. If upstreams trust a plain header, then any component that can reach a service can impersonate any tenant — and in a large estate something will eventually be able to reach a service that should not. The header is trivially forgeable precisely because it looks so convenient.

**Decision.** The gateway strips all client-supplied identity headers at admission without exception, resolves the principal itself, and injects a signed identity context — tenant, application, credential, scopes, plan tier, environment — over a mutually authenticated connection. Upstreams verify the signature and trust nothing else; raw client credentials never cross the boundary.

**How it is realised on Google Cloud.** Mutual TLS between the proxy and every upstream, with the identity context as a short-lived signed assertion carrying the request identifier so it cannot be replayed onto a different request. Header stripping is an admission-stage filter that runs before routing, so no configuration mistake can cause a client header to survive.

| Option | Verdict | Reasoning |
|---|---|---|
| Signed identity context over mTLS, client headers stripped | Chosen | One place verifies credentials; every service verifies one signature. Costs key distribution and rotation, and a signing step on the request path. |
| Plain trusted headers on a private network | Rejected | Universal, free, and it makes network reachability equivalent to authentication. It works until the first component that should not be able to reach a service can, and then it fails silently and totally. |
| Pass the client credential through for upstreams to verify | Rejected | Gives services full fidelity and duplicates the verification logic a hundred and twenty times, guaranteeing divergence. It also spreads live customer credentials across the entire estate. |

**What it buys**

- Adding a credential type is a gateway change, not an estate-wide migration — which is what makes ADR-03's hybrid affordable.
- An upstream compromise yields no callable customer credential, because none ever arrived.
- Scope enforcement is uniform, because it happens once rather than a hundred and twenty times with a hundred and twenty interpretations.

**What it costs**

- Signing keys to distribute and rotate, and a verification library every upstream must adopt and keep current.
- A signature operation on every request, inside a tight latency budget.
- Upstreams become dependent on the gateway's correctness for authorisation — a trust concentration that is the point, and is also a concentration.

**Choose differently when.** A service handling regulated data might legitimately require independent verification of the original credential as defence in depth. That is a per-service exception with a stated cost, not a reason to abandon the boundary for everything else.

**Why it holds up over time.** "Establish trust once, at the boundary, and prove it cryptographically downstream" is the shape every large estate converges on, whatever it is called this decade. The mechanism will change; the sentence will not.

> **Lesson.** A trust boundary that downstream components do not actually rely on is not a boundary — it is a suggestion, and it will be bypassed by the first component that finds it inconvenient.

### Identity at the edge

*How half a million credentials the platform did not issue are verified in single-digit milliseconds, and what happens when the thing that knows about them is unreachable.*

#### ADR-03 · Opaque credentials cached as verifiers, signed tokens verified statelessly, both under one denylist

**Status:** Accepted  ·  **Shown on views:** 08, 12, 21

*How is a credential verified in single-digit milliseconds when the store that knows about it may not be called?*

**Context.** The ecosystem has four kinds of caller and they do not want the same credential. Server-side integrations want a long-lived API key they can paste into an environment variable. OAuth clients hold short-lived bearer tokens. Partners sign assertions. A single scheme would be architecturally cleaner and would exclude most of the ecosystem. The tension is that stateless verification — check a signature, trust the claims — needs no lookup and cannot be revoked before expiry, while opaque credentials can be revoked instantly by the store that is not allowed to be called.

**Decision.** Two verification paths under one revocation mechanism. Opaque API keys are verified against a cached verifier held in the proxy, refreshed in the background with a 5-minute TTL. Signed tokens and partner assertions are verified in-process against cached issuer keys with no lookup at all. Both paths then consult the same pushed revocation denylist, which is what makes the stateless path revocable and the whole design consistent with ADR-01.

**How it is realised on Google Cloud.** The credential store in Spanner holds salted, computationally-hard verifiers — never secrets — and issuer public keys. A background refresher populates the proxy's cache; a Pub/Sub subscription maintains the denylist. A cache miss on an opaque key is the one credential-store call on the request path, budgeted at a p99 of 25 ms for that path only, and it is a read of a replica.

| Option | Verdict | Reasoning |
|---|---|---|
| Hybrid: cached verifiers plus stateless tokens, one denylist over both | Chosen | Serves the ecosystem as it is, and keeps one revocation story rather than two. Costs a denylist that must be distributed, bounded and expired. |
| Stateless signed tokens only | Rejected | The cleanest design and the right one for a closed system. Here it forces every integration onto a token exchange, which is a migration the ecosystem did not ask for, and revocation still needs a denylist — so the complexity is not actually avoided. |
| Opaque credentials only, with a lookup on every request | Rejected | Perfect revocation and an unmeetable latency budget, and it reintroduces exactly the dependency ADR-01 removes. It is what this design would collapse to if the denylist were abandoned. |

**What it buys**

- Existing integrations keep working unchanged, which is the difference between a gateway rollout and an ecosystem migration.
- One revocation mechanism covers every credential type, so a leaked partner assertion and a leaked API key are handled by the same operational path.
- Token lifetime becomes a tunable against the revocation SLO rather than a guess, since the denylist covers the gap.

**What it costs**

- Two verification code paths on the request path, both security-critical, both to be tested against the same denylist semantics.
- A cache-miss tail that is a property of the credential store, which is the one place ADR-01 is compromised in exchange for supporting opaque keys.
- The denylist grows with revocations and must expire entries with the credential they name.

**Choose differently when.** If the miss rate proved high enough to set the platform's p99 — the risk named in the one-pager — the answer is to shorten nothing and instead mint short-lived internal tokens at the boundary from a verified opaque key, converting a per-request miss into a per-session one. That is a larger change and is deliberately not in the MVP.

**Why it holds up over time.** Credential formats change every decade; the structure here does not. There will always be a long-lived secret somebody pasted into a configuration file and a short-lived token somebody minted, and there will always be a need to recall both. Separating verification from revocation is what lets either half be replaced.

> **Lesson.** Choosing a token lifetime is choosing a revocation latency. Teams reliably make the first choice for convenience and then discover they have made the second one.

#### ADR-04 · Fail open on read routes, fail closed on write routes

**Status:** Accepted  ·  **Shown on views:** 13, 20

*Past the credential cache TTL, with the identity dependency unreachable, does the gateway serve or refuse?*

**Context.** This is the decision every gateway design defers and every gateway incident forces. Fail closed and an identity outage becomes a total product outage for customers whose services are entirely healthy — the platform converts someone else's failure into its own, at maximum blast radius. Fail open and a credential revoked ten minutes ago keeps working for the length of the outage, which is unbounded because it is not a TTL. Answering it once for the whole platform means answering it wrongly for half the traffic, because a request that reads a shipment status and a request that moves money do not have the same cost of being wrong.

**Decision.** The posture is a property of the route, declared in configuration and defaulted by method. Read routes serve on stale cached verifiers for the duration of the outage. Write routes and any route marked sensitive refuse with a distinguishable 503 that names the degraded dependency. The pushed denylist is consulted in both cases and survives the outage, so a credential already known to be revoked is still refused.

**How it is realised on Google Cloud.** Each route carries a `degraded_posture` field in the configuration snapshot, defaulted to `closed` for non-idempotent methods and `open` for GET and HEAD, overridable per route by the API product owner. The degraded state is entered when the cache refresher has failed past a threshold and is exported as its own metric, so an operator can see the platform is running degraded before a customer tells them.

| Option | Verdict | Reasoning |
|---|---|---|
| Split by route consequence, defaulted by method | Chosen | Costs a per-route field and the discipline to set it. Buys the only outcome where neither the whole product nor the payment path is wrong. |
| Fail closed everywhere | Rejected | Correct for a bank, and it makes the identity provider a single point of failure for an entire SaaS. It is also the posture most likely to be quietly reverted under pressure during an incident, which is the worst way for it to change. |
| Fail open everywhere | Rejected | Maximises availability and means a revocation issued during an outage does nothing on the path where it matters most. Defensible only if every upstream independently authorises, which would duplicate the work this platform exists to centralise. |

**What it buys**

- An identity outage degrades the product rather than stopping it, and the part that keeps working is the part where being wrong is cheapest.
- The residual exposure is stated, bounded and owned, rather than discovered during the incident review.
- Upstreams on write paths can rely on the gateway's refusal, so they do not need to build a second authorisation layer for this case.

**What it costs**

- A per-route decision that must be set correctly across 800 routes, where the failure is silent and only visible during an outage nobody is having yet.
- Two behaviours to test, and a degraded mode that is exercised rarely — which means it must be exercised deliberately, in a game day.
- A customer whose read traffic works while their writes fail gets a confusing experience that support has to be able to explain.

**Choose differently when.** If the denylist proved able to cover effectively all revocations within the SLO, the fail-open window would carry almost no residual risk and read routes could stay open indefinitely with confidence. Conversely, a regulated tenant class could require closed everywhere, which is a per-tenant override rather than a redesign.

**Why it holds up over time.** Every system with an external dependency eventually faces this question, and the durable insight is that it has no single answer — it has an answer per consequence class. Splitting the posture by what being wrong costs transfers to any dependency, not just identity.

> **Lesson.** "Fail open or closed?" is the wrong question because it assumes one answer. The right question is which requests can afford to be wrong, and that is a property of what the request does, not of the dependency that failed.

### Fairness and limits

*Keeping one tenant's traffic from becoming another tenant's outage, without a distributed transaction on every request.*

#### ADR-05 · Local admission against a per-instance share, reconciled with a sharded shared counter

**Status:** Accepted  ·  **Shown on views:** 13, 14

*Where does the rate-limit counter live, given a 15 ms p99 budget and four scopes that must all be satisfied?*

**Context.** A fleet of stateless proxies has to enforce a limit that is defined globally. Purely local counters divided by fleet size are exact only when traffic is evenly distributed, which it never is, and they collapse the moment the fleet scales. A central counter consulted on every request is exact and adds a network round trip to the hot path, four times over if each scope is checked separately. And the counter store is a dependency whose outage must not remove the limits, because a rate limiter that fails open under load fails open exactly when the load is the problem.

**Decision.** Local buckets admit optimistically within a per-instance share of the limit and reconcile with a sharded shared counter in the background and on threshold crossings. All four scopes are evaluated in a single shared-counter round trip, not four. The shared counter carries a 5 ms timeout and no retry; on failure the proxy falls back to local buckets with a tightened ceiling rather than to no limit at all.

**How it is realised on Google Cloud.** Memorystore for Redis, sharded by principal so a hot tenant concentrates on its own shard rather than a shared one, with a Lua script evaluating all four scopes atomically in one call. Local buckets are in-process token buckets sized to the instance's share of the limit, adjusted as the fleet scales. Hard plan ceilings — the ones that must be exact — are checked against the shared counter without local admission, accepting the latency on the small fraction of traffic near a ceiling.

| Option | Verdict | Reasoning |
|---|---|---|
| Local admission with central reconciliation, one round trip for all scopes | Chosen | Bounds overshoot at ≤ 2% at p99 within the latency budget, and degrades to tighter local ceilings rather than to unlimited. Costs a genuinely approximate limit and a hot-path dependency. |
| Central counter consulted on every request | Rejected | Exact, and it makes the counter store's availability the platform's availability while spending a large share of the latency budget. Right when limits are contractual to the request rather than to the hour. |
| Purely local counters, limit divided by fleet size | Rejected | No dependency at all, and it is wrong whenever traffic is uneven or the fleet resizes — which is continuously. It under-limits a spread-out caller and over-limits a concentrated one, and neither error is visible. |

**What it buys**

- The limit decision costs one network round trip, not four, which is what makes four scopes affordable at all.
- A counter-store outage tightens limits instead of removing them, so the degraded mode is safe rather than permissive.
- Sharding by principal means a hot tenant's traffic concentrates on its own shard, which is the isolation requirement stated as a data-layout decision.

**What it costs**

- Limits are approximate, and the ≤ 2% overshoot figure is a claim that must be measured rather than asserted.
- Two counting mechanisms to reason about, and a reconciliation path that is subtle and easy to get wrong under fleet resize.
- Hard ceilings take the slower exact path, so a caller near its plan limit sees slightly higher latency than one nowhere near it.

**Choose differently when.** If a plan tier became contractual at the request level — a customer paying per call with an exact entitlement — local admission would be unacceptable for that tier and it would move entirely to the exact path, accepting the latency as the price of the contract.

**Why it holds up over time.** The trade between local speed and global accuracy is a permanent feature of distributed counting; the resolution here is to make the inaccuracy bounded, stated and degradable. That shape transfers to any shared budget — connections, tokens, spend — and outlives the particular store.

> **Lesson.** A distributed limit is always approximate. The design choice is not whether to be approximate but whether the error is bounded, stated and in the safe direction when the dependency fails.

#### ADR-06 · Billable usage is derived from access records, never from counters

**Status:** Accepted  ·  **Shown on views:** 10, 14

*The platform already counts every request for rate limiting. Should billing read those counters?*

**Context.** The temptation is obvious and the counters are right there. But ADR-05 deliberately made them approximate, in-region, ephemeral and explicitly safe to lose — those properties are what buy the latency budget and the degraded mode. Reading them for billing quietly promotes them to a financial system of record, at which point every availability decision about the counter store becomes a revenue decision, and the degraded mode that tightens limits during an outage also silently changes what a customer is charged.

**Decision.** Rate-limit counters are used for rate limiting and nothing else. Billable usage is derived from the durable access record stream, which has an RPO of 60 seconds, is replayable, and carries the request identifier, principal, route and outcome needed to explain a charge to the customer who disputes it.

**How it is realised on Google Cloud.** Access records are published to Pub/Sub and landed in BigQuery, partitioned by day and tenant. A daily rollup produces billable usage. The counter store has no export path to the billing pipeline at all, which is enforced by there being no such job rather than by a note in a runbook.

| Option | Verdict | Reasoning |
|---|---|---|
| Bill from durable access records | Chosen | Keeps the counters free to be approximate and losable. Costs a pipeline and a reconciliation window, and means billing is eventually consistent with usage by up to a day. |
| Bill from rate-limit counters | Rejected | Free and immediate, and it makes an ephemeral, deliberately lossy store into an auditable financial record. Every subsequent decision about that store becomes a finance conversation. |
| A third, separate metering counter on the hot path | Rejected | Accurate and durable and a second synchronous dependency on the request path, which ADR-01 exists to prevent. The access record already carries the same information asynchronously. |

**What it buys**

- Counters stay free to be approximate, in-region and losable, which is what makes the degraded mode in ADR-05 acceptable.
- A disputed charge can be traced to individual request identifiers, which is the only form of billing evidence a customer actually accepts.
- Billing survives a counter-store outage entirely, because it never read from it.

**What it costs**

- Billable usage lags real usage by up to a rollup period, so a customer's spend view is not live.
- The usage pipeline becomes revenue-critical and inherits a real RPO requirement, monitoring and replay tooling.
- Two numbers exist for "how many calls" — the limiter's and the bill's — and support must be able to explain why they differ.

**Choose differently when.** If a product required live spend enforcement — a hard budget cap that stops traffic mid-hour — the limiter would have to enforce a budget, and the reconciliation between the approximate enforcement and the exact bill would become a first-class design problem rather than an avoided one.

**Why it holds up over time.** The principle that a system of record must not be a component chosen for speed and designed to be losable is older than any of this technology and will outlast it. What changes is only which component is tempting this decade.

> **Lesson.** The moment a number is used to charge someone, its store inherits a durability requirement. Decide which numbers those are before someone writes the convenient query.

#### ADR-07 · Overload shedding is by plan tier, and every shed request is recorded

**Status:** Accepted  ·  **Shown on views:** 02, 14

*When the fleet is saturated beyond what limits alone can absorb, whose traffic is dropped?*

**Context.** Global overload is different from a tenant exceeding a quota: everyone is within their limits and there is still not enough capacity. Shedding uniformly is fair in the sense that a coin toss is fair, and it means an enterprise customer's production traffic fails at the same rate as a free-tier experiment. Not shedding at all means the queue grows until latency makes every caller's request useless, which is worse for everyone. And shedding silently means the affected developer sees unexplained failures, concludes the API is unreliable, and is not wrong.

**Decision.** Shedding is ordered by plan tier — free before paid, paid before enterprise — applied as a proportion within a tier rather than as a cliff, and every shed request produces an access record and a typed 503 that says the platform is shedding rather than that the request was invalid.

**How it is realised on Google Cloud.** A fleet saturation signal derived from connection count and CPU drives a shed proportion per tier in the proxy. The shed decision is a distinct response code path with its own metric series, so "we shed 4% of free-tier traffic for 90 seconds" is a sentence the platform can produce afterwards without reconstructing it.

| Option | Verdict | Reasoning |
|---|---|---|
| Tier-ordered proportional shedding, recorded | Chosen | Matches what the tiers were sold as. Costs an explicit policy that will occasionally have to be defended to a free-tier user, and a saturation signal that must be trustworthy. |
| Uniform random shedding | Rejected | Simplest and defensible in a system with no commercial tiers. Here it contradicts the product: paying for a higher tier should buy something during exactly the moments when it matters. |
| No shedding; queue and let latency rise | Rejected | Avoids the policy question and produces the worst outcome for everyone — timeouts, retries, amplification. Queueing is not an alternative to shedding, it is a slower way of shedding with less control. |

**What it buys**

- Plan tiers mean something at the moment they matter most, which is what the customer bought.
- A shed request is distinguishable from a rejected one in metrics and in the customer's own analytics, so nobody is debugging a phantom.
- The platform can state afterwards exactly who was affected and for how long.

**What it costs**

- An explicit, publishable policy that free-tier traffic is dropped first, which is a product decision as much as a technical one.
- A saturation signal that, if wrong, sheds traffic the platform could have served.
- More response paths to test, including one that only occurs under conditions that are hard to reproduce.

**Choose differently when.** If free-tier traffic were the top of the acquisition funnel and shedding it measurably cost signups, the ordering could be reversed for a bounded proportion — keeping a small free-tier allowance shedding last. That is a commercial decision the architecture can express, not a new mechanism.

**Why it holds up over time.** Any shared system eventually runs out of capacity, and the ordering of who loses is a values decision that the architecture will encode whether or not anyone states it. Stating it is what makes it revisable.

> **Lesson.** Every overloaded system sheds. The only question is whether the order was chosen or emerged from an implementation detail nobody meant as a policy.

### The published contract

*Versions, deprecation and retirement, for code the platform cannot refactor because other people wrote it.*

#### ADR-09 · Version in the path, one version per route, no gateway translation in v1

**Status:** Accepted  ·  **Shown on views:** 05, 11, 19

*Where does the API version live, and who is responsible for the difference between two versions?*

**Context.** Versioning in a header is architecturally tidier — the resource identifier stays stable — and it is invisible in a browser, in a log, in a curl command pasted into a support ticket, and to a developer trying to work out which version they are actually calling. Versioning in the path is cruder and is legible to everyone, including the person debugging at 2am. Separately, once two versions exist, something has to reconcile them: either each version routes to an implementation that owns its own behaviour, or the gateway transforms one into the other.

**Decision.** The version is a path segment. Each version of each route resolves to its own upstream group, and the implementation behind it owns its own contract. The gateway does not transform between versions in v1; it selects, validates and routes.

**How it is realised on Google Cloud.** Route matching is host, then path pattern including the version segment, then method. The configuration model makes `api_version` a first-class entity that owns routes, so a version is a unit that can be published, canaried, deprecated and sunset as a whole rather than route by route.

| Option | Verdict | Reasoning |
|---|---|---|
| Path versioning, no gateway transformation | Chosen | Legible everywhere and keeps compatibility logic next to the code that understands the domain. Costs duplication across versions in the upstream services. |
| Header or content-negotiated versioning | Rejected | Cleaner by REST principles and worse for humans. The cost lands on every developer debugging an integration, which is the population this platform is least able to help directly. |
| Gateway-side transformation from a single internal version | Rejected | Removes upstream duplication entirely and is genuinely attractive. Deferred to ADR-12 because it moves who is paged when a translation is wrong. |

**What it buys**

- A developer, a log line and a support ticket all agree on which version was called, with no inference.
- Two versions can be served by entirely different services, which is what lets a rewrite ship as v2 rather than as a risky in-place change.
- A version is a deployable, retirable unit, which is what makes view 19's lifecycle mechanical rather than aspirational.

**What it costs**

- Upstream services carry compatibility code for as long as the old version lives, which is a real and recurring tax.
- The published URL contains a version, so the resource identifier changes when the version does — a genuine REST compromise, made deliberately.
- More routes in configuration: 800 rather than a few hundred.

**Choose differently when.** If the number of live versions grew past what service teams could reasonably maintain — a long tail of pinned enterprise versions, say — gateway-side translation (ADR-12) becomes the cheaper option and this decision should be revisited rather than defended.

**Why it holds up over time.** Whatever protocol replaces today's, the property that matters is that the contract's identity is visible to the person debugging it. That is a human-factors constraint, not a technical one, and human factors change far more slowly than protocols.

> **Lesson.** Choose the version mechanism that is legible to the person who will be confused at 2am, not the one that is most correct in the specification. Most of the people it confuses do not work for you and cannot ask.

#### ADR-10 · Sunset is gated on measured active callers, with dated per-tenant pins

**Status:** Accepted  ·  **Shown on views:** 05, 19

*What actually permits an API version to be switched off?*

**Context.** The standard practice is to publish a date, send deprecation headers, and turn it off. The headers are machine-readable and largely unread; the date arrives; somebody's production integration breaks; and the platform learns who was still calling by breaking them. The opposite failure is just as common and more expensive: the date slips for everyone because two customers have not moved, then slips again, and the version becomes permanent — at which point the cost is carried forever by service teams maintaining compatibility code for an audience of two.

**Decision.** The platform refuses to sunset a version while it can observe active callers, unless an operator records an explicit override with their name on it. The platform reports active callers per version, by organisation, so the conversation is with named customers rather than with a number. Customers who cannot move receive a dated per-tenant version pin that expires; the sunset proceeds for everyone else on schedule.

**How it is realised on Google Cloud.** Active callers are derived from access records grouped by version and organisation over a trailing window, surfaced in the portal to the API product owner. `subscription.version_pin` carries an expiry, and the sunset check reads both. The override path writes to the audit ledger.

| Option | Verdict | Reasoning |
|---|---|---|
| Evidence-gated sunset with dated pins for the tail | Chosen | The date holds for the population while the exceptions are named, dated and visible. Costs a per-tenant routing exception and the discipline to let pins expire. |
| Date-driven sunset, headers only | Rejected | Simple and it is how most ecosystems do it, which is why most ecosystems have a story about the integration that broke. The platform has the caller data; not looking at it is a choice. |
| Never sunset; support every version indefinitely | Rejected | Maximum compatibility and unbounded cost, paid by every service team forever. It also removes the pressure that makes migrations happen at all. |

**What it buys**

- Nobody is surprised, because the platform knows who is still calling and can say so before the date rather than after.
- A version pin is visible, dated and owned, so long-tail compatibility ages out instead of becoming the steady state.
- The published date stays credible for the majority, which is what makes future deprecations believable.

**What it costs**

- Per-tenant routing exceptions in the configuration, each one a small permanent complexity until it expires.
- A social process — someone has to talk to the two customers — that the architecture can support but not replace.
- The active-caller query must be trustworthy; under-reporting it would turn this gate into false confidence.

**Choose differently when.** If per-tenant pins proliferated past a handful, the maintenance cost would exceed the migration cost and the right answer would become gateway-side translation (ADR-12) or a hard cut-off with commercial consequences.

**Why it holds up over time.** Any platform with external consumers faces the retirement problem, and the durable insight is that the decision needs evidence rather than a calendar. That is true of an HTTP API, a library, a file format or a wire protocol.

> **Lesson.** A deprecation policy without usage data is an announcement. The platform already knows who is still calling; the only question is whether anyone reads it before the date rather than after.

#### ADR-12 · Gateway-side version translation is deferred, and the reason is ownership

**Status:** Deferred to Phase 3  ·  **Shown on views:** 19

*Should the gateway transform a v1 request into the v2 contract so that only one implementation exists?*

**Context.** This is the most attractive rejected option in the package. Translation would let service teams delete their compatibility code, keep one implementation, and let the gateway present as many published versions as the ecosystem needs. The cost is not technical difficulty — the transformations are usually mechanical. The cost is that a wrong translation produces a wrong answer that looks like a correct response, and the team that gets paged is the platform team, who do not know what the field means, rather than the service team, who do.

**Decision.** Not in v1. Each published version routes to an implementation that owns its own contract (ADR-09). Translation is revisited in Phase 3, and only for a specific migration with a named owner, a differential test corpus, and agreement on who is paged when a transformed response is wrong.

**How it is realised on Google Cloud.** Nothing is built. The configuration model leaves room for it: a route already resolves to an upstream group independently of the published version, so translation can be inserted later without restructuring the routing table.

| Option | Verdict | Reasoning |
|---|---|---|
| Defer, keep compatibility with the implementation | Chosen | Keeps domain knowledge next to domain code and keeps the pager pointing at the team that can fix it. Costs real duplication in 120 services. |
| Translate at the gateway now | Rejected | Would remove the duplication and put semantic logic in the component with the widest blast radius and the least domain knowledge. It is the right answer for a small number of large, well-understood migrations — which is exactly the Phase 3 framing. |
| Translate in a dedicated adapter service per API | Rejected | A reasonable middle: keeps translation out of the gateway and gives it an owner. Rejected for v1 only because it is an extra hop and an extra service per API, which is a large cost to pay before any migration has proved it necessary. |

**What it buys**

- The gateway stays a policy and routing component, which keeps its blast radius proportional to what it knows.
- Compatibility bugs are diagnosed by people who understand the domain.
- The decision is recorded rather than forgotten, with the conditions for revisiting it written down.

**What it costs**

- Every service maintaining two live versions carries the duplication, and some of them will carry it badly.
- The number of live versions is limited by what service teams can maintain, which is what makes ADR-10's tail management necessary.
- A capability the platform could plausibly offer is not offered, and will be asked for.

**Choose differently when.** A single large migration affecting most of the ecosystem, where the transformation is provably mechanical and a differential test corpus exists, flips this — the duplication cost becomes concentrated and visible, which is exactly when centralising it pays.

**Why it holds up over time.** The general principle — do not put semantic knowledge in the component with the widest blast radius and the least context — outlives any particular protocol. Gateways accumulate logic by a series of individually reasonable additions, and this is the decision that names the criterion for refusing one.

> **Lesson.** Before centralising a capability, ask who gets paged when it is wrong. If the answer is a team that cannot diagnose it, the capability is in the wrong place however efficient it looks.

### Change and delivery

*How a change reaches a fleet that is serving, and how it is taken back when it is wrong.*

#### ADR-11 · Proxy filters are code, and ship through the same canary as configuration

**Status:** Accepted  ·  **Shown on views:** 07, 17

*Authentication, quota and routing logic run inside the proxy. Is that a deployment or a configuration change?*

**Context.** The resident-state design in ADR-01 moves a lot of logic into the proxy: credential verification, four-scope limit evaluation, version selection, header signing. That logic is code on the request path of every API call in the company. It is tempting to treat filter changes as low-risk because they are small and because the proxy is 'just' a proxy — and a filter regression is indistinguishable, from the customer's side, from the platform being down.

**Decision.** A filter change is a deployment and is subject to the same progressive rollout, health-gated scoring and automatic revert as a configuration change. The rollout mechanism does not distinguish them: both are a version that a subset of pods runs first.

**How it is realised on Google Cloud.** Filters are compiled into the proxy image; the image tag is part of the rollout artefact alongside the configuration version. The canary stage in view 17 runs a small set of pods on the new image carrying normal traffic, scored on the same signals, before any fleet-wide promotion.

| Option | Verdict | Reasoning |
|---|---|---|
| One rollout mechanism for configuration and filter code | Chosen | Removes the class of incident where a 'small' filter change bypassed the safety machinery. Costs a slower path for changes that genuinely are trivial. |
| Filters as dynamically loaded modules, hot-reloaded | Rejected | Fast and flexible, and it creates a second deployment path with weaker guarantees for the most safety-critical code in the platform. The flexibility is the danger. |
| Push logic into configuration as an expression language | Rejected | Avoids deployments and turns configuration into a programming language with no tests, no types and no review culture. Deferring risk into a DSL does not remove it, it hides it. |

**What it buys**

- The most dangerous code in the platform gets the strongest release discipline, rather than the weakest because it looked small.
- One rollout path to build, operate, test and trust.
- A filter regression is caught by canary scoring at 1% rather than discovered at 100%.

**What it costs**

- A trivial filter fix takes the full rollout time, which will be argued about the first time it is urgent.
- Image and configuration versions must be tracked together, or a revert reverts only half the change.
- A larger artefact to distribute than configuration alone.

**Choose differently when.** If an urgent security filter had to ship faster than the canary allows, the answer is a pre-approved expedited path with an explicit approver and a recorded exception — not a second, permanently weaker mechanism.

**Why it holds up over time.** The temptation to treat 'config-like' code as lower risk than 'real' code recurs in every platform and every era. The durable rule is that risk follows what runs on the request path, not what the artefact is called.

> **Lesson.** The question is never whether a change is code or configuration. It is whether it can break a live request — and if it can, it needs the same safety machinery whatever it is called.

#### ADR-13 · The rollout controller must be able to return insufficient evidence

**Status:** Accepted  ·  **Shown on views:** 05, 17

*A canary at 1% on a low-traffic route sees no errors. Is that healthy?*

**Context.** Progressive delivery rests on an assumption nobody states: that the canary window contains enough requests for the absence of failures to mean something. On a route taking 5,000 requests a second, a 1% canary over five minutes is 150,000 requests and silence is strong evidence. On a route taking two requests a minute, the same window is one request, and silence is nothing at all. Every rollout system that defaults to healthy ships untested changes to exactly the routes that are least exercised — and those are disproportionately the routes that matter to one large customer.

**Decision.** The canary verdict has three values, not two: healthy, failing, and insufficient evidence. A window that did not contain enough requests to detect a regression at the configured sensitivity returns the third, and the rollout holds rather than promoting. Holding escalates: extend the window, raise the weight, inject synthetic traffic, or require a human gate — a choice made per route, in advance.

**How it is realised on Google Cloud.** The rollout controller computes the minimum request count needed to detect the configured effect size at the configured confidence, from the route's own observed baseline, and compares it with what the window actually contained. The verdict, including the count, is written to the rollout row so it is visible afterwards.

| Option | Verdict | Reasoning |
|---|---|---|
| Three-valued verdict with an explicit hold | Chosen | Makes the limit of the mechanism visible instead of papering over it. Costs slower rollouts on quiet routes and a statistics conversation the team must actually have. |
| Two-valued verdict, default healthy | Rejected | The industry default. It is why progressive delivery so often fails to catch the regression it was bought to catch, and the failure is silent — the dashboard says the canary passed. |
| Fixed minimum request count per canary stage | Rejected | Simple and much better than nothing, and it is wrong in both directions — too slow for high-traffic routes, still too permissive for very quiet ones, because the right threshold depends on the route's own error baseline. |

**What it buys**

- A quiet route cannot ship an untested change under cover of a green canary.
- The rollout record says how much evidence a promotion actually rested on, which is the first question in a post-incident review.
- Sensitivity becomes an explicit per-route setting rather than an accident of traffic volume.

**What it costs**

- Rollouts on low-traffic routes take longer or need a human, which will be resented.
- The controller needs a statistical model and a per-route baseline, which is more than a threshold comparison.
- A third verdict is a third path in every downstream tool and dashboard.

**Choose differently when.** If synthetic traffic could be generated safely for every route — hard for write routes with real side effects — every canary could be given enough evidence and the hold state would become rare rather than routine.

**Why it holds up over time.** The gap between 'no failures observed' and 'no failures' is permanent, and it widens as systems are split into more, smaller, quieter routes. Any automated gate that cannot say 'I do not know' will eventually say 'fine' about something it never looked at.

> **Lesson.** An automated verdict with two values will use the safe-sounding one for the case it cannot judge. Give it a third value, or it will lie to you politely.

### Running it

*Topology, scaling signals, evidence and the data the platform refuses to hold.*

#### ADR-14 · Two regions active–active behind one anycast VIP, each sized for the whole load

**Status:** Accepted  ·  **Shown on views:** 13, 16

*What topology delivers a 120 s client-visible recovery from losing a region?*

**Context.** Active–passive is cheaper and the failover path is exercised for the first time on the day it is needed, by people under pressure, using a runbook written when the system was different. DNS-based failover cannot meet a 120 s promise, because resolver and client caches are outside the platform's control and routinely ignore short TTLs. And a standby region sized at half capacity discovers, at failover, that it is now the entire platform at half the necessary size.

**Decision.** Both regions serve live traffic continuously behind a single global anycast VIP. Each is provisioned to carry the entire global load at burst. Evacuation is the load balancer withdrawing an unhealthy region, not an operator action, and the receiving region needs no scaling event to absorb the traffic.

**How it is realised on Google Cloud.** Global external Application Load Balancer with one anycast IP and health checks against every regional backend. Three zones per region: two serving, one of headroom. Capacity is sized against the 5× burst assumption so that evacuation and a burst can coincide without a scaling race.

| Option | Verdict | Reasoning |
|---|---|---|
| Active–active, anycast, each region full-size | Chosen | The failover path is the normal path, so it is exercised continuously. Costs roughly 2.4× the compute of a single-region design once headroom is counted. |
| Active–passive with DNS failover | Rejected | Much cheaper and cannot meet a 120 s RTO, because client resolver caches are not the platform's to control. Also leaves the passive region's correctness unverified until the incident. |
| Active–active with each region at 60% capacity | Rejected | Saves real money and only works if the failure is graceful enough to leave time for a scaling event. Region failures are rarely that considerate, and a scaling race during an evacuation is a second incident. |

**What it buys**

- Evacuation is a health-check outcome with no human in the loop and no DNS in the path.
- The receiving region is already serving real traffic, so its correctness is continuously proven rather than assumed.
- Zonal failure is absorbed by standing headroom with no client-visible effect at all.

**What it costs**

- Roughly 2.4× single-region compute cost, which ask.md requires be reported so the availability target can be renegotiated against its price.
- Two regional counter stores, so a caller whose traffic is split across regions is limited approximately per region rather than globally — an acknowledged limit of ADR-05.
- Data residency constraints interact badly with anycast and are deferred to Phase 3.

**Choose differently when.** If the availability target were relaxed to 99.9%, or if cost pressure became acute, a warm standby at partial capacity with an explicit scaling step would be defensible — at an honest RTO of minutes rather than seconds, stated as such.

**Why it holds up over time.** The principle that a recovery path must be exercised continuously rather than rehearsed occasionally is as old as operations and is repeatedly rediscovered. Anycast will be replaced by something; 'the failover path should be the normal path' will not.

> **Lesson.** A failover path that is only used during failures is a path that has never worked. Make it the normal path, or accept that the recovery time is a guess.

#### ADR-15 · Autoscale on connection count and CPU saturation, not request rate

**Status:** Accepted  ·  **Shown on views:** 13, 16

*Which signal should add capacity to the proxy fleet?*

**Context.** Request rate is the obvious signal and it is wrong for a proxy. Two workloads at identical request rates can have entirely different costs: ten thousand long-lived connections issuing occasional requests consume memory and file descriptors while barely touching CPU; a burst of new TLS handshakes consumes CPU dramatically while the request count looks unremarkable. A fleet scaled on request rate is under-provisioned for a connection-heavy caller and over-provisioned for a throughput-heavy one, and it discovers the difference during an incident.

**Decision.** Capacity is driven by connection count and CPU saturation together, with request rate used for observability rather than for scaling. Sufficient standing headroom is maintained to absorb the 5× burst assumption for 120 s without a scaling event having completed, because scaling is slower than a burst.

**How it is realised on Google Cloud.** Horizontal pod autoscaling on custom metrics — open connections per pod and CPU utilisation — with a floor set by the evacuation requirement in ADR-14 rather than by current traffic. The 10,000 requests/s per instance at ≤ 70% CPU assumption is what converts a load target into a pod count.

| Option | Verdict | Reasoning |
|---|---|---|
| Connections and CPU, with standing burst headroom | Chosen | Matches what actually exhausts a proxy. Costs headroom that is idle most of the time, which is the price of not scaling during the burst. |
| Request rate | Rejected | The intuitive choice and the one that mis-sizes both connection-heavy and handshake-heavy traffic. It is right only when the workload mix is uniform, which an open ecosystem's never is. |
| Latency-driven scaling | Rejected | Scales on the symptom rather than the cause, so it necessarily acts after customers have felt it. Useful as an alarm, wrong as a control signal. |

**What it buys**

- The fleet is sized by what actually exhausts it, so saturation is predicted rather than discovered.
- A connection-heavy partner cannot quietly exhaust file descriptors while every rate dashboard looks healthy.
- Burst absorption does not depend on a scaling event completing in time.

**What it costs**

- Idle headroom that is visible in the bill every month and defended every quarter.
- Two signals to tune, with the interaction between them needing periodic revisiting as the traffic mix changes.
- Request rate remains the number everyone intuitively reaches for, so the scaling behaviour will need explaining repeatedly.

**Choose differently when.** If the traffic mix became genuinely uniform — a single SDK, a single connection pattern — request rate would become an adequate proxy for cost and the simpler signal would be defensible.

**Why it holds up over time.** The lesson that the scaling signal must be the resource that actually runs out applies to every kind of server. What changes is which resource that is; the method of finding out does not.

> **Lesson.** Scale on what runs out, not on what is easy to count. They are the same number only when the workload is uniform, and an open ecosystem's never is.

#### ADR-16 · Telemetry is sampled; policy decisions never are

**Status:** Accepted  ·  **Shown on views:** 09, 18

*At ten billion requests a day, what is kept?*

**Context.** Keeping a full-fidelity record of every request is the honest default and, at roughly a kilobyte each, it is about ten terabytes a day before compression — a cost that competes directly with the compute serving the traffic. Sampling uniformly is the standard answer and it quietly destroys the platform's ability to answer the questions it exists to answer: whether a specific customer's credential was refused, why, and how often. The rare events are the valuable ones, and uniform sampling is precisely the strategy that discards them.

**Decision.** Sampling is by outcome class, not uniform. Errors are kept at full fidelity. Successes are sampled at a per-route rate. Authentication and authorisation decisions, configuration changes and shed decisions are never sampled at any volume. Under pipeline backpressure the platform samples down further and then drops — but the never-sampled classes are the last to go and are backed by their own path.

**How it is realised on Google Cloud.** The telemetry filter tags each record with its class at emission. The Pub/Sub publisher applies the per-class policy and the backpressure ladder in-process, without a network call. Audit-class events are published to a separate topic with its own retention and its own alerting, so a saturated analytics pipeline cannot lose them.

| Option | Verdict | Reasoning |
|---|---|---|
| Class-based sampling with a never-sampled audit set | Chosen | Keeps the rare, valuable events and pays only for the common, redundant ones. Costs a policy that must be maintained as new classes appear. |
| Uniform sampling at a fixed rate | Rejected | Simplest and statistically defensible for aggregate metrics, and it makes per-customer investigation impossible — the single most common reason anyone queries this data. |
| Keep everything | Rejected | Best for investigation and a storage cost that competes with the serving fleet. Revisit if storage economics change enough to make the question uninteresting. |

**What it buys**

- "Why was this customer refused?" is always answerable, which is the question support actually receives.
- Telemetry cost scales with error rate rather than with traffic, which is the direction that matches its value.
- The audit path survives analytics-pipeline saturation because it does not share it.

**What it costs**

- Aggregate success metrics derived from samples need their rate applied correctly, and someone will forget.
- A sampling policy to maintain per route and per class, with a new class arriving every time a feature does.
- Two telemetry paths to operate rather than one.

**Choose differently when.** If per-request storage became cheap enough to be uninteresting, keeping everything is simpler and strictly better, and this decision should be deleted rather than defended.

**Why it holds up over time.** The insight that the valuable observations are the rare ones, and that uniform sampling is therefore the wrong default, holds for any telemetry system at any scale. It gets more true as volume grows, not less.

> **Lesson.** Uniform sampling optimises for the events you already understand. Sample by how much an event would tell you, not by how often it happens.

#### ADR-17 · One shared fleet in v1; cells are deferred with the trigger named

**Status:** Deferred to Phase 3  ·  **Shown on views:** 07, 16

*Should the proxy fleet be partitioned into cells per plan tier or per large tenant?*

**Context.** A single global fleet means a single global blast radius. One bad configuration version, one filter regression or one hot tenant affects every customer at once, and every mitigation in this design — canary, tier shedding, counter sharding — narrows the window rather than the population. Cells would bound the population directly. They also multiply fixed cost per cell, require tenant-to-cell resolution at an anycast edge without adding a hop, and turn every operational action into an action repeated N times.

**Decision.** One shared fleet in v1 and Phase 2, with blast radius managed by time rather than by population: progressive rollout, automatic revert, per-tenant counter sharding and tier-aware shedding. Cells are deferred with two explicit triggers recorded now: the first incident whose blast radius would have been contained by cells, or the first contractual requirement for isolation — whichever arrives first.

**How it is realised on Google Cloud.** Nothing is built. The configuration model already carries plan tier on the principal and residency on the organisation, so a future cell assignment has somewhere to live. Cell cost is estimated in advance so the decision, when it comes, is not made under incident pressure.

| Option | Verdict | Reasoning |
|---|---|---|
| Shared fleet now, cells deferred with named triggers | Chosen | Avoids paying for isolation before there is evidence it is needed, while making sure the decision is taken deliberately rather than after an outage. |
| Cells per plan tier from day one | Rejected | The right end state for a mature platform, and it multiplies cost and operational surface before any traffic has proved the need. Premature cellularisation is as real as premature optimisation and much more expensive. |
| Dedicated fleets for the largest tenants only | Rejected | A reasonable middle and the likely first step when a trigger fires. Rejected for v1 because it starts the operational multiplication for a benefit no incident has yet demonstrated. |

**What it buys**

- Lower cost and a single operational surface while the platform is establishing itself.
- The deferral is recorded with its triggers, so it is a decision rather than an omission.
- The data model does not preclude cells, so adopting them is an addition rather than a restructuring.

**What it costs**

- A genuine single global blast radius for the whole of v1 and Phase 2, which is the largest accepted risk in this package.
- Enterprise customers asking for isolation will be told 'not yet', which is a commercial cost.
- Data residency cannot be honoured properly without something cell-shaped, which is why it is deferred alongside.

**Choose differently when.** Either named trigger flips it. The first is an incident whose population would have been bounded by cells; the second is a contract that requires isolation. Neither requires new analysis, which is the point of naming them now.

**Why it holds up over time.** Every shared platform eventually confronts the trade between the efficiency of sharing and the blast radius of sharing, and the answer moves as the platform's customers change. Recording the trigger rather than the verdict is what makes the decision reviewable years later.

> **Lesson.** Deferring a decision is legitimate. Deferring it without writing down what would change your mind is just forgetting it slowly.

#### ADR-18 · No request or response bodies are logged; field capture is opt-in

**Status:** Accepted  ·  **Shown on views:** 10, 18

*The gateway sees every payload in the product. How much of it is kept?*

**Context.** Body logging is the single most requested debugging feature a gateway team receives, and it is entirely reasonable each time it is asked for. It is also how the component with the widest reach in the estate becomes the largest concentration of customer data in the company — payment details, personal data, everything any API carries — sitting in a log store with a different access model from the databases those fields came from, and a retention nobody chose deliberately. The failure is not the logging; it is that the decision gets made once, per debugging session, by whoever is on call.

**Decision.** Request and response bodies are never captured. Headers are captured by per-field opt-in rather than opt-out, so a new header added next year is not logged by default. Access records carry identifiers, outcomes and latencies, which is what is needed to explain a response without containing one. Response caching, which does hold payloads, is per route, explicit, and bounded by TTL.

**How it is realised on Google Cloud.** The telemetry filter has no body-capture path at all — it is absent rather than disabled, so it cannot be enabled under pressure by a configuration change. Header capture reads an allow-list from the configuration snapshot. Debugging that genuinely requires payloads happens in the sandbox environment, where the payloads are the developer's own.

| Option | Verdict | Reasoning |
|---|---|---|
| No body capture; per-field header opt-in | Chosen | Removes the largest breach surface the platform could create. Costs real debugging capability, permanently and visibly. |
| Body capture, enabled per route when needed | Rejected | Enormously useful and it means the decision is taken during an incident by whoever is on call, with retention nobody chose. Every large payload-logging incident started as a temporary measure. |
| Body capture with automated redaction | Rejected | The sophisticated answer, and it makes the platform's data safety depend on a redaction rule set keeping pace with 120 teams' schema changes. It will not. |

**What it buys**

- The gateway is not a secondary copy of the product's most sensitive data, so its compromise is a control failure rather than a data breach.
- Retention questions for the access-record store are simple, because there is nothing sensitive in it.
- A new header or field added by any team is not silently captured.

**What it costs**

- Genuinely harder debugging of payload-shaped problems, in production, permanently.
- The sandbox becomes load-bearing for reproduction, and it must be good enough that it can be.
- The feature will be requested repeatedly, and each refusal has to be re-argued with someone who has a real problem.

**Choose differently when.** A regulated obligation to retain full message content — some financial messaging regimes require it — would force capture, and it would then belong in a separate, encrypted, access-controlled store with its own retention and its own audit, not in the access-record stream.

**Why it holds up over time.** Data minimisation at the widest-reaching component is a durable rule. What changes is which component is widest-reaching this decade; the reasoning transfers to whatever replaces the gateway.

> **Lesson.** The most dangerous data store is the one nobody decided to create. Make the dangerous capability absent rather than disabled, because a disabled feature is one incident away from being enabled.

## Every package used, in one table

The nouns this package uses in a particular way. Where a term is ordinary but the role it plays here is not, the role is what is written down.

| Package | What it is | What it does here | Considered instead |
|---|---|---|---|
| Request plane | The proxy fleet and the state resident inside it: configuration snapshot, credential cache, revocation denylist, local counters. | The only thing that must be available for the product's API to work. Its independence from everything else is the architecture. | Calling it the data plane, which invites confusion with the data the API carries. |
| Configuration version | An immutable, signed, monotonically numbered snapshot of routes, upstream groups, policies, products and plans. | The unit of change, of rollout, of revert and of attribution. Every access record names the version that produced it. | Treating configuration as mutable settings, which makes "what was running?" unanswerable and revert a re-run. |
| Resident state | Everything the proxy needs to decide a request, held in-process and refreshed in the background. | The mechanism by which ADR-01 is true. If a decision needs something not in resident state, either the state grows or the decision does not belong on the request path. | A cache, which is the same data with none of the guarantees — a cache has a cold path, and cold paths correlate with incidents. |
| Principal | The resolved identity of a caller: tenant, application, credential, granted scopes, plan tier, environment. | Established once at the boundary and carried downstream as a signed assertion. Everything after admission reasons about the principal, never about the credential. | Passing the credential through, which makes every upstream a verifier and spreads live secrets across the estate. |
| Scope (of a limit) | One of four levels a request is counted against: credential, application, tenant, route. | All four must pass, and the rejection names which one failed. A 429 that does not say which scope was hit is an error the caller cannot act on. | A single tenant-level limit, which cannot protect a fragile upstream from one well-behaved tenant. |
| API product | The durable published thing a developer subscribes to, owning versions, plans and scopes. | The entity that persists while versions come and go. Ownership, pricing and entitlement attach here, not to a version that will be retired and take the relationship with it. | Subscribing to routes, which cannot express a version pin or a plan. |
| Degraded posture | A per-route declaration of what happens when the identity dependency is unreachable past the cache TTL: serve, or refuse. | The field that turns "fail open or closed?" from an argument into a configuration decision made ahead of the incident, per consequence class. | A platform-wide setting, which is wrong for half the traffic whichever way it is set. |
| Insufficient evidence | A canary verdict distinct from both healthy and failing, returned when the observation window held too few requests to distinguish a regression from noise. | The state that stops a low-traffic route's rollout reporting success because nothing happened. Without it, progressive delivery is theatre on exactly the routes least exercised. | Defaulting to healthy, which is the industry norm and is how untested changes reach 100%. |
| Access record | The structured, durable per-request fact: identifiers, principal, route, version, variant, outcome, both latencies, config version. | The system of record for usage, billing, developer analytics and canary scoring. Deliberately carries no payload. | Deriving these from counters or from upstream logs, neither of which can attribute a response to the configuration that produced it. |
| Coverage | The measured proportion of proxy instances confirmed to hold a given configuration or denylist version. | What turns a propagation SLO into something falsifiable. A 30 s revocation promise without a coverage figure is a sentence in a document. | Assuming a push succeeded, which is the assumption every stale-cache incident is made of. |