Architecture Views

When the gateway needs to know a route, a credential, a scope, a limit or a version, may it look that up while a request is waiting?

Context

Every capability this platform has naturally wants a lookup. Authentication wants the credential store. Authorisation wants the grant table. Rate limiting wants the plan. Routing wants the route table. Each lookup is individually reasonable and individually fast, and together they make the product's availability the product of five dependencies' availabilities, with the gateway's latency set by whichever of them is slowest at the 99th percentile. The failure mode is not gradual: a control-plane deploy that adds 40 ms to a config read becomes a 40 ms regression on every API call in the company, and a control-plane outage becomes a total outage of a product whose services are all healthy.

Decision

Every per-request decision is made from state already resident in the proxy process: a signed configuration snapshot, a credential verifier cache, a pushed revocation denylist, and local counters. The control plane, the configuration store, the credential database and the product catalogue are never called during a request. They are refreshed asynchronously, in the background, on their own schedule.

How it is realised on Google Cloud

Envoy pods on GKE hold the configuration snapshot in memory, pulled from a signed object in a dual-region Cloud Storage bucket and refreshed on a poll. The credential cache and denylist are populated by a background refresher and a Pub/Sub subscription respectively. Spanner, the Config API on Cloud Run and the product catalogue have no ingress path from the request-serving path at all — enforced by network policy and by service-account scoping, not by convention.

Options weighed

• ChosenResident state only; the control plane is refreshed asynchronously: Makes the control plane's availability target legitimately lower than the data plane's, and puts a hard ceiling on gateway-added latency. Costs eventual consistency on every kind of change, which is why revocation becomes ADR-03 and propagation becomes a measured SLO.
• RejectedLookup with an aggressive cache: This is what most gateways do and it works until the cache misses during the incident. A cache is a latency optimisation; it is not an availability boundary, because its cold path is still a dependency and cold paths correlate with outages.
• RejectedA sidecar that owns the lookups: Moves the network hop rather than removing it, and adds a second process whose failure modes are now on the request path. Attractive because it looks like isolation; it is really indirection.

Consequences

What it buys

• A control-plane outage stops change and changes nothing about traffic — which is what makes a 99.9% control-plane target defensible against a 99.99% data-plane target.
• Gateway-added p99 contains no control-plane round trip, so the latency budget is a property of the proxy rather than of the slowest management service.
• Every dependency failure on view 13 degrades a dimension instead of availability, because the dependencies were removed from the path before the failure analysis started.

What it costs

• Every change is eventually consistent, and each one needs a stated propagation SLO plus the machinery to measure it.
• Revocation cannot be immediate, which pushes real complexity into ADR-03 and leaves a stated exposure window.
• Proxy memory becomes a sizing constraint, because configuration and credential state are resident rather than remote.

Choose differently when

If the credential population grew past what can be held in each proxy — tens of millions of verifiers rather than hundreds of thousands — resident credential state would stop being viable and the design would have to move to signed, self-describing tokens with a small denylist, making ADR-03 the primary decision and this one its consequence.

Why it holds up over time

The rule names no technology. It is a statement about which dependencies may sit on a request, and it survives replacing the proxy, the cloud and the credential format. It is also the property a future team would most easily erode — one convenient lookup for one urgent feature — which is why it is drawn as a boundary on the component view rather than written down as a guideline.

How does a routing or policy change reach a serving fleet, and what exactly is reverted when it is wrong?

Context

ADR-01 means the fleet runs on cached state, and cached state that nobody can identify is unmanageable: "which proxies have the new limit?" has no answer, "revert" means re-running a pipeline and hoping, and two proxies can silently disagree about the same request. The alternative failure is equally common: a streaming push protocol that works beautifully until the control plane is down at the moment a pod restarts, and the pod comes up with no configuration at all — turning a management outage into a serving outage through the back door.

Decision

A change is committed as an immutable, monotonically versioned, signed configuration snapshot before it is distributed. Proxies pull the snapshot, verify the signature, and report the version they are actually running. A rollout is a decision about which version a set of proxies should be on; a revert is selecting an earlier version, which is already published and already verified.

How it is realised on Google Cloud

The Config API writes the change to Spanner as a new version row with a digest and a parent version. A publisher renders the snapshot, signs it, and writes it to a dual-region Cloud Storage bucket under its version. Proxies poll for the version they have been told to run, verify, and load. Version N−1 is always retained. Each pod reports its resident version on a heartbeat, so fleet coverage is a table rather than an assumption.

Options weighed

• ChosenImmutable signed snapshots, pulled by the data plane: A pod restarting during a control-plane outage can still fetch a known-good snapshot from object storage, because object storage is the dependency, not the control plane. Costs a propagation delay measured in seconds.
• RejectedStreaming push from the control plane (xDS-style): Faster and the industry default, but it makes the control plane a runtime dependency of every pod start, which is precisely the coupling ADR-01 removes. Right for a mesh, where the control plane is already in the failure domain.
• RejectedConfiguration baked into the container image: Maximum immutability, and it makes a limit change a fleet redeploy. A platform where changing one tenant's quota takes a rolling restart will grow a side channel within a month, and the side channel will not be versioned.

Consequences

What it buys

• Revert is a selection, bounded by how fast a pod can load a snapshot, and the thing being reverted to is already known to parse.
• "What was this fleet running when that request failed?" is answerable, because access records carry the config version (view 11).
• Progressive rollout of configuration becomes possible at all, since a subset of pods can be told to run a different version.

What it costs

• A propagation window in which the fleet is split across two versions, which has to be bounded and alerted rather than wished away.
• Snapshot rendering and signing is a build step with its own failure modes, and a stale publisher silently freezes the platform's ability to change.
• Storage and retention of every version, and the discipline to never mutate one.

Choose differently when

If propagation latency had to drop to sub-second — an adaptive policy reacting to live traffic, say — pull would stop being viable and a streaming channel would be needed, accepting the control-plane coupling and compensating with a resident fallback snapshot for cold starts.

Why it holds up over time

Immutable, content-addressed, signed artefacts with an explicit version are how every durable delivery system ends up working, from package managers to container registries. The shape outlives the storage technology, and the discipline it enforces — that the deployed thing has a name — is what makes every later operational question answerable.

How does an upstream service know who is calling, and how much is it required to re-verify?

Context

If upstreams re-authenticate the caller, the platform has centralised nothing: every service needs credential-verification code, every credential type change is a hundred-and-twenty-service migration, and the services will diverge in how strictly they enforce scopes. If upstreams trust a plain header, then any component that can reach a service can impersonate any tenant — and in a large estate something will eventually be able to reach a service that should not. The header is trivially forgeable precisely because it looks so convenient.

Decision

The gateway strips all client-supplied identity headers at admission without exception, resolves the principal itself, and injects a signed identity context — tenant, application, credential, scopes, plan tier, environment — over a mutually authenticated connection. Upstreams verify the signature and trust nothing else; raw client credentials never cross the boundary.

How it is realised on Google Cloud

Mutual TLS between the proxy and every upstream, with the identity context as a short-lived signed assertion carrying the request identifier so it cannot be replayed onto a different request. Header stripping is an admission-stage filter that runs before routing, so no configuration mistake can cause a client header to survive.

Options weighed

• ChosenSigned identity context over mTLS, client headers stripped: One place verifies credentials; every service verifies one signature. Costs key distribution and rotation, and a signing step on the request path.
• RejectedPlain trusted headers on a private network: Universal, free, and it makes network reachability equivalent to authentication. It works until the first component that should not be able to reach a service can, and then it fails silently and totally.
• RejectedPass the client credential through for upstreams to verify: Gives services full fidelity and duplicates the verification logic a hundred and twenty times, guaranteeing divergence. It also spreads live customer credentials across the entire estate.

Consequences

What it buys

• Adding a credential type is a gateway change, not an estate-wide migration — which is what makes ADR-03's hybrid affordable.
• An upstream compromise yields no callable customer credential, because none ever arrived.
• Scope enforcement is uniform, because it happens once rather than a hundred and twenty times with a hundred and twenty interpretations.

What it costs

• Signing keys to distribute and rotate, and a verification library every upstream must adopt and keep current.
• A signature operation on every request, inside a tight latency budget.
• Upstreams become dependent on the gateway's correctness for authorisation — a trust concentration that is the point, and is also a concentration.

Choose differently when

A service handling regulated data might legitimately require independent verification of the original credential as defence in depth. That is a per-service exception with a stated cost, not a reason to abandon the boundary for everything else.

Why it holds up over time

"Establish trust once, at the boundary, and prove it cryptographically downstream" is the shape every large estate converges on, whatever it is called this decade. The mechanism will change; the sentence will not.

How is a credential verified in single-digit milliseconds when the store that knows about it may not be called?

Context

The ecosystem has four kinds of caller and they do not want the same credential. Server-side integrations want a long-lived API key they can paste into an environment variable. OAuth clients hold short-lived bearer tokens. Partners sign assertions. A single scheme would be architecturally cleaner and would exclude most of the ecosystem. The tension is that stateless verification — check a signature, trust the claims — needs no lookup and cannot be revoked before expiry, while opaque credentials can be revoked instantly by the store that is not allowed to be called.

Decision

Two verification paths under one revocation mechanism. Opaque API keys are verified against a cached verifier held in the proxy, refreshed in the background with a 5-minute TTL. Signed tokens and partner assertions are verified in-process against cached issuer keys with no lookup at all. Both paths then consult the same pushed revocation denylist, which is what makes the stateless path revocable and the whole design consistent with ADR-01.

How it is realised on Google Cloud

The credential store in Spanner holds salted, computationally-hard verifiers — never secrets — and issuer public keys. A background refresher populates the proxy's cache; a Pub/Sub subscription maintains the denylist. A cache miss on an opaque key is the one credential-store call on the request path, budgeted at a p99 of 25 ms for that path only, and it is a read of a replica.

Options weighed

• ChosenHybrid: cached verifiers plus stateless tokens, one denylist over both: Serves the ecosystem as it is, and keeps one revocation story rather than two. Costs a denylist that must be distributed, bounded and expired.
• RejectedStateless signed tokens only: The cleanest design and the right one for a closed system. Here it forces every integration onto a token exchange, which is a migration the ecosystem did not ask for, and revocation still needs a denylist — so the complexity is not actually avoided.
• RejectedOpaque credentials only, with a lookup on every request: Perfect revocation and an unmeetable latency budget, and it reintroduces exactly the dependency ADR-01 removes. It is what this design would collapse to if the denylist were abandoned.

Consequences

What it buys

• Existing integrations keep working unchanged, which is the difference between a gateway rollout and an ecosystem migration.
• One revocation mechanism covers every credential type, so a leaked partner assertion and a leaked API key are handled by the same operational path.
• Token lifetime becomes a tunable against the revocation SLO rather than a guess, since the denylist covers the gap.

What it costs

• Two verification code paths on the request path, both security-critical, both to be tested against the same denylist semantics.
• A cache-miss tail that is a property of the credential store, which is the one place ADR-01 is compromised in exchange for supporting opaque keys.
• The denylist grows with revocations and must expire entries with the credential they name.

Choose differently when

If the miss rate proved high enough to set the platform's p99 — the risk named in the one-pager — the answer is to shorten nothing and instead mint short-lived internal tokens at the boundary from a verified opaque key, converting a per-request miss into a per-session one. That is a larger change and is deliberately not in the MVP.

Why it holds up over time

Credential formats change every decade; the structure here does not. There will always be a long-lived secret somebody pasted into a configuration file and a short-lived token somebody minted, and there will always be a need to recall both. Separating verification from revocation is what lets either half be replaced.

Past the credential cache TTL, with the identity dependency unreachable, does the gateway serve or refuse?

Context

This is the decision every gateway design defers and every gateway incident forces. Fail closed and an identity outage becomes a total product outage for customers whose services are entirely healthy — the platform converts someone else's failure into its own, at maximum blast radius. Fail open and a credential revoked ten minutes ago keeps working for the length of the outage, which is unbounded because it is not a TTL. Answering it once for the whole platform means answering it wrongly for half the traffic, because a request that reads a shipment status and a request that moves money do not have the same cost of being wrong.

Decision

The posture is a property of the route, declared in configuration and defaulted by method. Read routes serve on stale cached verifiers for the duration of the outage. Write routes and any route marked sensitive refuse with a distinguishable 503 that names the degraded dependency. The pushed denylist is consulted in both cases and survives the outage, so a credential already known to be revoked is still refused.

How it is realised on Google Cloud

Each route carries a `degraded_posture` field in the configuration snapshot, defaulted to `closed` for non-idempotent methods and `open` for GET and HEAD, overridable per route by the API product owner. The degraded state is entered when the cache refresher has failed past a threshold and is exported as its own metric, so an operator can see the platform is running degraded before a customer tells them.

Options weighed

• ChosenSplit by route consequence, defaulted by method: Costs a per-route field and the discipline to set it. Buys the only outcome where neither the whole product nor the payment path is wrong.
• RejectedFail closed everywhere: Correct for a bank, and it makes the identity provider a single point of failure for an entire SaaS. It is also the posture most likely to be quietly reverted under pressure during an incident, which is the worst way for it to change.
• RejectedFail open everywhere: Maximises availability and means a revocation issued during an outage does nothing on the path where it matters most. Defensible only if every upstream independently authorises, which would duplicate the work this platform exists to centralise.

Consequences

What it buys

• An identity outage degrades the product rather than stopping it, and the part that keeps working is the part where being wrong is cheapest.
• The residual exposure is stated, bounded and owned, rather than discovered during the incident review.
• Upstreams on write paths can rely on the gateway's refusal, so they do not need to build a second authorisation layer for this case.

What it costs

• A per-route decision that must be set correctly across 800 routes, where the failure is silent and only visible during an outage nobody is having yet.
• Two behaviours to test, and a degraded mode that is exercised rarely — which means it must be exercised deliberately, in a game day.
• A customer whose read traffic works while their writes fail gets a confusing experience that support has to be able to explain.

Choose differently when

If the denylist proved able to cover effectively all revocations within the SLO, the fail-open window would carry almost no residual risk and read routes could stay open indefinitely with confidence. Conversely, a regulated tenant class could require closed everywhere, which is a per-tenant override rather than a redesign.

Why it holds up over time

Every system with an external dependency eventually faces this question, and the durable insight is that it has no single answer — it has an answer per consequence class. Splitting the posture by what being wrong costs transfers to any dependency, not just identity.

Where does the rate-limit counter live, given a 15 ms p99 budget and four scopes that must all be satisfied?

Context

A fleet of stateless proxies has to enforce a limit that is defined globally. Purely local counters divided by fleet size are exact only when traffic is evenly distributed, which it never is, and they collapse the moment the fleet scales. A central counter consulted on every request is exact and adds a network round trip to the hot path, four times over if each scope is checked separately. And the counter store is a dependency whose outage must not remove the limits, because a rate limiter that fails open under load fails open exactly when the load is the problem.

Decision

Local buckets admit optimistically within a per-instance share of the limit and reconcile with a sharded shared counter in the background and on threshold crossings. All four scopes are evaluated in a single shared-counter round trip, not four. The shared counter carries a 5 ms timeout and no retry; on failure the proxy falls back to local buckets with a tightened ceiling rather than to no limit at all.

How it is realised on Google Cloud

Memorystore for Redis, sharded by principal so a hot tenant concentrates on its own shard rather than a shared one, with a Lua script evaluating all four scopes atomically in one call. Local buckets are in-process token buckets sized to the instance's share of the limit, adjusted as the fleet scales. Hard plan ceilings — the ones that must be exact — are checked against the shared counter without local admission, accepting the latency on the small fraction of traffic near a ceiling.

Options weighed

• ChosenLocal admission with central reconciliation, one round trip for all scopes: Bounds overshoot at ≤ 2% at p99 within the latency budget, and degrades to tighter local ceilings rather than to unlimited. Costs a genuinely approximate limit and a hot-path dependency.
• RejectedCentral counter consulted on every request: Exact, and it makes the counter store's availability the platform's availability while spending a large share of the latency budget. Right when limits are contractual to the request rather than to the hour.
• RejectedPurely local counters, limit divided by fleet size: No dependency at all, and it is wrong whenever traffic is uneven or the fleet resizes — which is continuously. It under-limits a spread-out caller and over-limits a concentrated one, and neither error is visible.

Consequences

What it buys

• The limit decision costs one network round trip, not four, which is what makes four scopes affordable at all.
• A counter-store outage tightens limits instead of removing them, so the degraded mode is safe rather than permissive.
• Sharding by principal means a hot tenant's traffic concentrates on its own shard, which is the isolation requirement stated as a data-layout decision.

What it costs

• Limits are approximate, and the ≤ 2% overshoot figure is a claim that must be measured rather than asserted.
• Two counting mechanisms to reason about, and a reconciliation path that is subtle and easy to get wrong under fleet resize.
• Hard ceilings take the slower exact path, so a caller near its plan limit sees slightly higher latency than one nowhere near it.

Choose differently when

If a plan tier became contractual at the request level — a customer paying per call with an exact entitlement — local admission would be unacceptable for that tier and it would move entirely to the exact path, accepting the latency as the price of the contract.

Why it holds up over time

The trade between local speed and global accuracy is a permanent feature of distributed counting; the resolution here is to make the inaccuracy bounded, stated and degradable. That shape transfers to any shared budget — connections, tokens, spend — and outlives the particular store.

The platform already counts every request for rate limiting. Should billing read those counters?

Context

The temptation is obvious and the counters are right there. But ADR-05 deliberately made them approximate, in-region, ephemeral and explicitly safe to lose — those properties are what buy the latency budget and the degraded mode. Reading them for billing quietly promotes them to a financial system of record, at which point every availability decision about the counter store becomes a revenue decision, and the degraded mode that tightens limits during an outage also silently changes what a customer is charged.

Decision

Rate-limit counters are used for rate limiting and nothing else. Billable usage is derived from the durable access record stream, which has an RPO of 60 seconds, is replayable, and carries the request identifier, principal, route and outcome needed to explain a charge to the customer who disputes it.

How it is realised on Google Cloud

Access records are published to Pub/Sub and landed in BigQuery, partitioned by day and tenant. A daily rollup produces billable usage. The counter store has no export path to the billing pipeline at all, which is enforced by there being no such job rather than by a note in a runbook.

Options weighed

• ChosenBill from durable access records: Keeps the counters free to be approximate and losable. Costs a pipeline and a reconciliation window, and means billing is eventually consistent with usage by up to a day.
• RejectedBill from rate-limit counters: Free and immediate, and it makes an ephemeral, deliberately lossy store into an auditable financial record. Every subsequent decision about that store becomes a finance conversation.
• RejectedA third, separate metering counter on the hot path: Accurate and durable and a second synchronous dependency on the request path, which ADR-01 exists to prevent. The access record already carries the same information asynchronously.

Consequences

What it buys

• Counters stay free to be approximate, in-region and losable, which is what makes the degraded mode in ADR-05 acceptable.
• A disputed charge can be traced to individual request identifiers, which is the only form of billing evidence a customer actually accepts.
• Billing survives a counter-store outage entirely, because it never read from it.

What it costs

• Billable usage lags real usage by up to a rollup period, so a customer's spend view is not live.
• The usage pipeline becomes revenue-critical and inherits a real RPO requirement, monitoring and replay tooling.
• Two numbers exist for "how many calls" — the limiter's and the bill's — and support must be able to explain why they differ.

Choose differently when

If a product required live spend enforcement — a hard budget cap that stops traffic mid-hour — the limiter would have to enforce a budget, and the reconciliation between the approximate enforcement and the exact bill would become a first-class design problem rather than an avoided one.

Why it holds up over time

The principle that a system of record must not be a component chosen for speed and designed to be losable is older than any of this technology and will outlast it. What changes is only which component is tempting this decade.

When the fleet is saturated beyond what limits alone can absorb, whose traffic is dropped?

Context

Global overload is different from a tenant exceeding a quota: everyone is within their limits and there is still not enough capacity. Shedding uniformly is fair in the sense that a coin toss is fair, and it means an enterprise customer's production traffic fails at the same rate as a free-tier experiment. Not shedding at all means the queue grows until latency makes every caller's request useless, which is worse for everyone. And shedding silently means the affected developer sees unexplained failures, concludes the API is unreliable, and is not wrong.

Decision

Shedding is ordered by plan tier — free before paid, paid before enterprise — applied as a proportion within a tier rather than as a cliff, and every shed request produces an access record and a typed 503 that says the platform is shedding rather than that the request was invalid.

How it is realised on Google Cloud

A fleet saturation signal derived from connection count and CPU drives a shed proportion per tier in the proxy. The shed decision is a distinct response code path with its own metric series, so "we shed 4% of free-tier traffic for 90 seconds" is a sentence the platform can produce afterwards without reconstructing it.

Options weighed

• ChosenTier-ordered proportional shedding, recorded: Matches what the tiers were sold as. Costs an explicit policy that will occasionally have to be defended to a free-tier user, and a saturation signal that must be trustworthy.
• RejectedUniform random shedding: Simplest and defensible in a system with no commercial tiers. Here it contradicts the product: paying for a higher tier should buy something during exactly the moments when it matters.
• RejectedNo shedding; queue and let latency rise: Avoids the policy question and produces the worst outcome for everyone — timeouts, retries, amplification. Queueing is not an alternative to shedding, it is a slower way of shedding with less control.

Consequences

What it buys

• Plan tiers mean something at the moment they matter most, which is what the customer bought.
• A shed request is distinguishable from a rejected one in metrics and in the customer's own analytics, so nobody is debugging a phantom.
• The platform can state afterwards exactly who was affected and for how long.

What it costs

• An explicit, publishable policy that free-tier traffic is dropped first, which is a product decision as much as a technical one.
• A saturation signal that, if wrong, sheds traffic the platform could have served.
• More response paths to test, including one that only occurs under conditions that are hard to reproduce.

Choose differently when

If free-tier traffic were the top of the acquisition funnel and shedding it measurably cost signups, the ordering could be reversed for a bounded proportion — keeping a small free-tier allowance shedding last. That is a commercial decision the architecture can express, not a new mechanism.

Why it holds up over time

Any shared system eventually runs out of capacity, and the ordering of who loses is a values decision that the architecture will encode whether or not anyone states it. Stating it is what makes it revisable.

Where does the API version live, and who is responsible for the difference between two versions?

Context

Versioning in a header is architecturally tidier — the resource identifier stays stable — and it is invisible in a browser, in a log, in a curl command pasted into a support ticket, and to a developer trying to work out which version they are actually calling. Versioning in the path is cruder and is legible to everyone, including the person debugging at 2am. Separately, once two versions exist, something has to reconcile them: either each version routes to an implementation that owns its own behaviour, or the gateway transforms one into the other.

Decision

The version is a path segment. Each version of each route resolves to its own upstream group, and the implementation behind it owns its own contract. The gateway does not transform between versions in v1; it selects, validates and routes.

How it is realised on Google Cloud

Route matching is host, then path pattern including the version segment, then method. The configuration model makes `api_version` a first-class entity that owns routes, so a version is a unit that can be published, canaried, deprecated and sunset as a whole rather than route by route.

Options weighed

• ChosenPath versioning, no gateway transformation: Legible everywhere and keeps compatibility logic next to the code that understands the domain. Costs duplication across versions in the upstream services.
• RejectedHeader or content-negotiated versioning: Cleaner by REST principles and worse for humans. The cost lands on every developer debugging an integration, which is the population this platform is least able to help directly.
• RejectedGateway-side transformation from a single internal version: Removes upstream duplication entirely and is genuinely attractive. Deferred to ADR-12 because it moves who is paged when a translation is wrong.

Consequences

What it buys

• A developer, a log line and a support ticket all agree on which version was called, with no inference.
• Two versions can be served by entirely different services, which is what lets a rewrite ship as v2 rather than as a risky in-place change.
• A version is a deployable, retirable unit, which is what makes view 19's lifecycle mechanical rather than aspirational.

What it costs

• Upstream services carry compatibility code for as long as the old version lives, which is a real and recurring tax.
• The published URL contains a version, so the resource identifier changes when the version does — a genuine REST compromise, made deliberately.
• More routes in configuration: 800 rather than a few hundred.

Choose differently when

If the number of live versions grew past what service teams could reasonably maintain — a long tail of pinned enterprise versions, say — gateway-side translation (ADR-12) becomes the cheaper option and this decision should be revisited rather than defended.

Why it holds up over time

Whatever protocol replaces today's, the property that matters is that the contract's identity is visible to the person debugging it. That is a human-factors constraint, not a technical one, and human factors change far more slowly than protocols.

What actually permits an API version to be switched off?

Context

The standard practice is to publish a date, send deprecation headers, and turn it off. The headers are machine-readable and largely unread; the date arrives; somebody's production integration breaks; and the platform learns who was still calling by breaking them. The opposite failure is just as common and more expensive: the date slips for everyone because two customers have not moved, then slips again, and the version becomes permanent — at which point the cost is carried forever by service teams maintaining compatibility code for an audience of two.

Decision

The platform refuses to sunset a version while it can observe active callers, unless an operator records an explicit override with their name on it. The platform reports active callers per version, by organisation, so the conversation is with named customers rather than with a number. Customers who cannot move receive a dated per-tenant version pin that expires; the sunset proceeds for everyone else on schedule.

How it is realised on Google Cloud

Active callers are derived from access records grouped by version and organisation over a trailing window, surfaced in the portal to the API product owner. `subscription.version_pin` carries an expiry, and the sunset check reads both. The override path writes to the audit ledger.

Options weighed

• ChosenEvidence-gated sunset with dated pins for the tail: The date holds for the population while the exceptions are named, dated and visible. Costs a per-tenant routing exception and the discipline to let pins expire.
• RejectedDate-driven sunset, headers only: Simple and it is how most ecosystems do it, which is why most ecosystems have a story about the integration that broke. The platform has the caller data; not looking at it is a choice.
• RejectedNever sunset; support every version indefinitely: Maximum compatibility and unbounded cost, paid by every service team forever. It also removes the pressure that makes migrations happen at all.

Consequences

What it buys

• Nobody is surprised, because the platform knows who is still calling and can say so before the date rather than after.
• A version pin is visible, dated and owned, so long-tail compatibility ages out instead of becoming the steady state.
• The published date stays credible for the majority, which is what makes future deprecations believable.

What it costs

• Per-tenant routing exceptions in the configuration, each one a small permanent complexity until it expires.
• A social process — someone has to talk to the two customers — that the architecture can support but not replace.
• The active-caller query must be trustworthy; under-reporting it would turn this gate into false confidence.

Choose differently when

If per-tenant pins proliferated past a handful, the maintenance cost would exceed the migration cost and the right answer would become gateway-side translation (ADR-12) or a hard cut-off with commercial consequences.

Why it holds up over time

Any platform with external consumers faces the retirement problem, and the durable insight is that the decision needs evidence rather than a calendar. That is true of an HTTP API, a library, a file format or a wire protocol.

Should the gateway transform a v1 request into the v2 contract so that only one implementation exists?

Context

This is the most attractive rejected option in the package. Translation would let service teams delete their compatibility code, keep one implementation, and let the gateway present as many published versions as the ecosystem needs. The cost is not technical difficulty — the transformations are usually mechanical. The cost is that a wrong translation produces a wrong answer that looks like a correct response, and the team that gets paged is the platform team, who do not know what the field means, rather than the service team, who do.

Decision

Not in v1. Each published version routes to an implementation that owns its own contract (ADR-09). Translation is revisited in Phase 3, and only for a specific migration with a named owner, a differential test corpus, and agreement on who is paged when a transformed response is wrong.

How it is realised on Google Cloud

Nothing is built. The configuration model leaves room for it: a route already resolves to an upstream group independently of the published version, so translation can be inserted later without restructuring the routing table.

Options weighed

• ChosenDefer, keep compatibility with the implementation: Keeps domain knowledge next to domain code and keeps the pager pointing at the team that can fix it. Costs real duplication in 120 services.
• RejectedTranslate at the gateway now: Would remove the duplication and put semantic logic in the component with the widest blast radius and the least domain knowledge. It is the right answer for a small number of large, well-understood migrations — which is exactly the Phase 3 framing.
• RejectedTranslate in a dedicated adapter service per API: A reasonable middle: keeps translation out of the gateway and gives it an owner. Rejected for v1 only because it is an extra hop and an extra service per API, which is a large cost to pay before any migration has proved it necessary.

Consequences

What it buys

• The gateway stays a policy and routing component, which keeps its blast radius proportional to what it knows.
• Compatibility bugs are diagnosed by people who understand the domain.
• The decision is recorded rather than forgotten, with the conditions for revisiting it written down.

What it costs

• Every service maintaining two live versions carries the duplication, and some of them will carry it badly.
• The number of live versions is limited by what service teams can maintain, which is what makes ADR-10's tail management necessary.
• A capability the platform could plausibly offer is not offered, and will be asked for.

Choose differently when

A single large migration affecting most of the ecosystem, where the transformation is provably mechanical and a differential test corpus exists, flips this — the duplication cost becomes concentrated and visible, which is exactly when centralising it pays.

Why it holds up over time

The general principle — do not put semantic knowledge in the component with the widest blast radius and the least context — outlives any particular protocol. Gateways accumulate logic by a series of individually reasonable additions, and this is the decision that names the criterion for refusing one.

Authentication, quota and routing logic run inside the proxy. Is that a deployment or a configuration change?

Context

The resident-state design in ADR-01 moves a lot of logic into the proxy: credential verification, four-scope limit evaluation, version selection, header signing. That logic is code on the request path of every API call in the company. It is tempting to treat filter changes as low-risk because they are small and because the proxy is 'just' a proxy — and a filter regression is indistinguishable, from the customer's side, from the platform being down.

Decision

A filter change is a deployment and is subject to the same progressive rollout, health-gated scoring and automatic revert as a configuration change. The rollout mechanism does not distinguish them: both are a version that a subset of pods runs first.

How it is realised on Google Cloud

Filters are compiled into the proxy image; the image tag is part of the rollout artefact alongside the configuration version. The canary stage in view 17 runs a small set of pods on the new image carrying normal traffic, scored on the same signals, before any fleet-wide promotion.

Options weighed

• ChosenOne rollout mechanism for configuration and filter code: Removes the class of incident where a 'small' filter change bypassed the safety machinery. Costs a slower path for changes that genuinely are trivial.
• RejectedFilters as dynamically loaded modules, hot-reloaded: Fast and flexible, and it creates a second deployment path with weaker guarantees for the most safety-critical code in the platform. The flexibility is the danger.
• RejectedPush logic into configuration as an expression language: Avoids deployments and turns configuration into a programming language with no tests, no types and no review culture. Deferring risk into a DSL does not remove it, it hides it.

Consequences

What it buys

• The most dangerous code in the platform gets the strongest release discipline, rather than the weakest because it looked small.
• One rollout path to build, operate, test and trust.
• A filter regression is caught by canary scoring at 1% rather than discovered at 100%.

What it costs

• A trivial filter fix takes the full rollout time, which will be argued about the first time it is urgent.
• Image and configuration versions must be tracked together, or a revert reverts only half the change.
• A larger artefact to distribute than configuration alone.

Choose differently when

If an urgent security filter had to ship faster than the canary allows, the answer is a pre-approved expedited path with an explicit approver and a recorded exception — not a second, permanently weaker mechanism.

Why it holds up over time

The temptation to treat 'config-like' code as lower risk than 'real' code recurs in every platform and every era. The durable rule is that risk follows what runs on the request path, not what the artefact is called.

A canary at 1% on a low-traffic route sees no errors. Is that healthy?

Context

Progressive delivery rests on an assumption nobody states: that the canary window contains enough requests for the absence of failures to mean something. On a route taking 5,000 requests a second, a 1% canary over five minutes is 150,000 requests and silence is strong evidence. On a route taking two requests a minute, the same window is one request, and silence is nothing at all. Every rollout system that defaults to healthy ships untested changes to exactly the routes that are least exercised — and those are disproportionately the routes that matter to one large customer.

Decision

The canary verdict has three values, not two: healthy, failing, and insufficient evidence. A window that did not contain enough requests to detect a regression at the configured sensitivity returns the third, and the rollout holds rather than promoting. Holding escalates: extend the window, raise the weight, inject synthetic traffic, or require a human gate — a choice made per route, in advance.

How it is realised on Google Cloud

The rollout controller computes the minimum request count needed to detect the configured effect size at the configured confidence, from the route's own observed baseline, and compares it with what the window actually contained. The verdict, including the count, is written to the rollout row so it is visible afterwards.

Options weighed

• ChosenThree-valued verdict with an explicit hold: Makes the limit of the mechanism visible instead of papering over it. Costs slower rollouts on quiet routes and a statistics conversation the team must actually have.
• RejectedTwo-valued verdict, default healthy: The industry default. It is why progressive delivery so often fails to catch the regression it was bought to catch, and the failure is silent — the dashboard says the canary passed.
• RejectedFixed minimum request count per canary stage: Simple and much better than nothing, and it is wrong in both directions — too slow for high-traffic routes, still too permissive for very quiet ones, because the right threshold depends on the route's own error baseline.

Consequences

What it buys

• A quiet route cannot ship an untested change under cover of a green canary.
• The rollout record says how much evidence a promotion actually rested on, which is the first question in a post-incident review.
• Sensitivity becomes an explicit per-route setting rather than an accident of traffic volume.

What it costs

• Rollouts on low-traffic routes take longer or need a human, which will be resented.
• The controller needs a statistical model and a per-route baseline, which is more than a threshold comparison.
• A third verdict is a third path in every downstream tool and dashboard.

Choose differently when

If synthetic traffic could be generated safely for every route — hard for write routes with real side effects — every canary could be given enough evidence and the hold state would become rare rather than routine.

Why it holds up over time

The gap between 'no failures observed' and 'no failures' is permanent, and it widens as systems are split into more, smaller, quieter routes. Any automated gate that cannot say 'I do not know' will eventually say 'fine' about something it never looked at.

What topology delivers a 120 s client-visible recovery from losing a region?

Context

Active–passive is cheaper and the failover path is exercised for the first time on the day it is needed, by people under pressure, using a runbook written when the system was different. DNS-based failover cannot meet a 120 s promise, because resolver and client caches are outside the platform's control and routinely ignore short TTLs. And a standby region sized at half capacity discovers, at failover, that it is now the entire platform at half the necessary size.

Decision

Both regions serve live traffic continuously behind a single global anycast VIP. Each is provisioned to carry the entire global load at burst. Evacuation is the load balancer withdrawing an unhealthy region, not an operator action, and the receiving region needs no scaling event to absorb the traffic.

How it is realised on Google Cloud

Global external Application Load Balancer with one anycast IP and health checks against every regional backend. Three zones per region: two serving, one of headroom. Capacity is sized against the 5× burst assumption so that evacuation and a burst can coincide without a scaling race.

Options weighed

• ChosenActive–active, anycast, each region full-size: The failover path is the normal path, so it is exercised continuously. Costs roughly 2.4× the compute of a single-region design once headroom is counted.
• RejectedActive–passive with DNS failover: Much cheaper and cannot meet a 120 s RTO, because client resolver caches are not the platform's to control. Also leaves the passive region's correctness unverified until the incident.
• RejectedActive–active with each region at 60% capacity: Saves real money and only works if the failure is graceful enough to leave time for a scaling event. Region failures are rarely that considerate, and a scaling race during an evacuation is a second incident.

Consequences

What it buys

• Evacuation is a health-check outcome with no human in the loop and no DNS in the path.
• The receiving region is already serving real traffic, so its correctness is continuously proven rather than assumed.
• Zonal failure is absorbed by standing headroom with no client-visible effect at all.

What it costs

• Roughly 2.4× single-region compute cost, which ask.md requires be reported so the availability target can be renegotiated against its price.
• Two regional counter stores, so a caller whose traffic is split across regions is limited approximately per region rather than globally — an acknowledged limit of ADR-05.
• Data residency constraints interact badly with anycast and are deferred to Phase 3.

Choose differently when

If the availability target were relaxed to 99.9%, or if cost pressure became acute, a warm standby at partial capacity with an explicit scaling step would be defensible — at an honest RTO of minutes rather than seconds, stated as such.

Why it holds up over time

The principle that a recovery path must be exercised continuously rather than rehearsed occasionally is as old as operations and is repeatedly rediscovered. Anycast will be replaced by something; 'the failover path should be the normal path' will not.

Which signal should add capacity to the proxy fleet?

Context

Request rate is the obvious signal and it is wrong for a proxy. Two workloads at identical request rates can have entirely different costs: ten thousand long-lived connections issuing occasional requests consume memory and file descriptors while barely touching CPU; a burst of new TLS handshakes consumes CPU dramatically while the request count looks unremarkable. A fleet scaled on request rate is under-provisioned for a connection-heavy caller and over-provisioned for a throughput-heavy one, and it discovers the difference during an incident.

Decision

Capacity is driven by connection count and CPU saturation together, with request rate used for observability rather than for scaling. Sufficient standing headroom is maintained to absorb the 5× burst assumption for 120 s without a scaling event having completed, because scaling is slower than a burst.

How it is realised on Google Cloud

Horizontal pod autoscaling on custom metrics — open connections per pod and CPU utilisation — with a floor set by the evacuation requirement in ADR-14 rather than by current traffic. The 10,000 requests/s per instance at ≤ 70% CPU assumption is what converts a load target into a pod count.

Options weighed

• ChosenConnections and CPU, with standing burst headroom: Matches what actually exhausts a proxy. Costs headroom that is idle most of the time, which is the price of not scaling during the burst.
• RejectedRequest rate: The intuitive choice and the one that mis-sizes both connection-heavy and handshake-heavy traffic. It is right only when the workload mix is uniform, which an open ecosystem's never is.
• RejectedLatency-driven scaling: Scales on the symptom rather than the cause, so it necessarily acts after customers have felt it. Useful as an alarm, wrong as a control signal.

Consequences

What it buys

• The fleet is sized by what actually exhausts it, so saturation is predicted rather than discovered.
• A connection-heavy partner cannot quietly exhaust file descriptors while every rate dashboard looks healthy.
• Burst absorption does not depend on a scaling event completing in time.

What it costs

• Idle headroom that is visible in the bill every month and defended every quarter.
• Two signals to tune, with the interaction between them needing periodic revisiting as the traffic mix changes.
• Request rate remains the number everyone intuitively reaches for, so the scaling behaviour will need explaining repeatedly.

Choose differently when

If the traffic mix became genuinely uniform — a single SDK, a single connection pattern — request rate would become an adequate proxy for cost and the simpler signal would be defensible.

Why it holds up over time

The lesson that the scaling signal must be the resource that actually runs out applies to every kind of server. What changes is which resource that is; the method of finding out does not.

At ten billion requests a day, what is kept?

Context

Keeping a full-fidelity record of every request is the honest default and, at roughly a kilobyte each, it is about ten terabytes a day before compression — a cost that competes directly with the compute serving the traffic. Sampling uniformly is the standard answer and it quietly destroys the platform's ability to answer the questions it exists to answer: whether a specific customer's credential was refused, why, and how often. The rare events are the valuable ones, and uniform sampling is precisely the strategy that discards them.

Decision

Sampling is by outcome class, not uniform. Errors are kept at full fidelity. Successes are sampled at a per-route rate. Authentication and authorisation decisions, configuration changes and shed decisions are never sampled at any volume. Under pipeline backpressure the platform samples down further and then drops — but the never-sampled classes are the last to go and are backed by their own path.

How it is realised on Google Cloud

The telemetry filter tags each record with its class at emission. The Pub/Sub publisher applies the per-class policy and the backpressure ladder in-process, without a network call. Audit-class events are published to a separate topic with its own retention and its own alerting, so a saturated analytics pipeline cannot lose them.

Options weighed

• ChosenClass-based sampling with a never-sampled audit set: Keeps the rare, valuable events and pays only for the common, redundant ones. Costs a policy that must be maintained as new classes appear.
• RejectedUniform sampling at a fixed rate: Simplest and statistically defensible for aggregate metrics, and it makes per-customer investigation impossible — the single most common reason anyone queries this data.
• RejectedKeep everything: Best for investigation and a storage cost that competes with the serving fleet. Revisit if storage economics change enough to make the question uninteresting.

Consequences

What it buys

• "Why was this customer refused?" is always answerable, which is the question support actually receives.
• Telemetry cost scales with error rate rather than with traffic, which is the direction that matches its value.
• The audit path survives analytics-pipeline saturation because it does not share it.

What it costs

• Aggregate success metrics derived from samples need their rate applied correctly, and someone will forget.
• A sampling policy to maintain per route and per class, with a new class arriving every time a feature does.
• Two telemetry paths to operate rather than one.

Choose differently when

If per-request storage became cheap enough to be uninteresting, keeping everything is simpler and strictly better, and this decision should be deleted rather than defended.

Why it holds up over time

The insight that the valuable observations are the rare ones, and that uniform sampling is therefore the wrong default, holds for any telemetry system at any scale. It gets more true as volume grows, not less.

Should the proxy fleet be partitioned into cells per plan tier or per large tenant?

Context

A single global fleet means a single global blast radius. One bad configuration version, one filter regression or one hot tenant affects every customer at once, and every mitigation in this design — canary, tier shedding, counter sharding — narrows the window rather than the population. Cells would bound the population directly. They also multiply fixed cost per cell, require tenant-to-cell resolution at an anycast edge without adding a hop, and turn every operational action into an action repeated N times.

Decision

One shared fleet in v1 and Phase 2, with blast radius managed by time rather than by population: progressive rollout, automatic revert, per-tenant counter sharding and tier-aware shedding. Cells are deferred with two explicit triggers recorded now: the first incident whose blast radius would have been contained by cells, or the first contractual requirement for isolation — whichever arrives first.

How it is realised on Google Cloud

Nothing is built. The configuration model already carries plan tier on the principal and residency on the organisation, so a future cell assignment has somewhere to live. Cell cost is estimated in advance so the decision, when it comes, is not made under incident pressure.

Options weighed

• ChosenShared fleet now, cells deferred with named triggers: Avoids paying for isolation before there is evidence it is needed, while making sure the decision is taken deliberately rather than after an outage.
• RejectedCells per plan tier from day one: The right end state for a mature platform, and it multiplies cost and operational surface before any traffic has proved the need. Premature cellularisation is as real as premature optimisation and much more expensive.
• RejectedDedicated fleets for the largest tenants only: A reasonable middle and the likely first step when a trigger fires. Rejected for v1 because it starts the operational multiplication for a benefit no incident has yet demonstrated.

Consequences

What it buys

• Lower cost and a single operational surface while the platform is establishing itself.
• The deferral is recorded with its triggers, so it is a decision rather than an omission.
• The data model does not preclude cells, so adopting them is an addition rather than a restructuring.

What it costs

• A genuine single global blast radius for the whole of v1 and Phase 2, which is the largest accepted risk in this package.
• Enterprise customers asking for isolation will be told 'not yet', which is a commercial cost.
• Data residency cannot be honoured properly without something cell-shaped, which is why it is deferred alongside.

Choose differently when

Either named trigger flips it. The first is an incident whose population would have been bounded by cells; the second is a contract that requires isolation. Neither requires new analysis, which is the point of naming them now.

Why it holds up over time

Every shared platform eventually confronts the trade between the efficiency of sharing and the blast radius of sharing, and the answer moves as the platform's customers change. Recording the trigger rather than the verdict is what makes the decision reviewable years later.

The gateway sees every payload in the product. How much of it is kept?

Context

Body logging is the single most requested debugging feature a gateway team receives, and it is entirely reasonable each time it is asked for. It is also how the component with the widest reach in the estate becomes the largest concentration of customer data in the company — payment details, personal data, everything any API carries — sitting in a log store with a different access model from the databases those fields came from, and a retention nobody chose deliberately. The failure is not the logging; it is that the decision gets made once, per debugging session, by whoever is on call.

Decision

Request and response bodies are never captured. Headers are captured by per-field opt-in rather than opt-out, so a new header added next year is not logged by default. Access records carry identifiers, outcomes and latencies, which is what is needed to explain a response without containing one. Response caching, which does hold payloads, is per route, explicit, and bounded by TTL.

How it is realised on Google Cloud

The telemetry filter has no body-capture path at all — it is absent rather than disabled, so it cannot be enabled under pressure by a configuration change. Header capture reads an allow-list from the configuration snapshot. Debugging that genuinely requires payloads happens in the sandbox environment, where the payloads are the developer's own.

Options weighed

• ChosenNo body capture; per-field header opt-in: Removes the largest breach surface the platform could create. Costs real debugging capability, permanently and visibly.
• RejectedBody capture, enabled per route when needed: Enormously useful and it means the decision is taken during an incident by whoever is on call, with retention nobody chose. Every large payload-logging incident started as a temporary measure.
• RejectedBody capture with automated redaction: The sophisticated answer, and it makes the platform's data safety depend on a redaction rule set keeping pace with 120 teams' schema changes. It will not.

Consequences

What it buys

• The gateway is not a secondary copy of the product's most sensitive data, so its compromise is a control failure rather than a data breach.
• Retention questions for the access-record store are simple, because there is nothing sensitive in it.
• A new header or field added by any team is not silently captured.

What it costs

• Genuinely harder debugging of payload-shaped problems, in production, permanently.
• The sandbox becomes load-bearing for reproduction, and it must be good enough that it can be.
• The feature will be requested repeatedly, and each refusal has to be re-argued with someone who has a real problem.

Choose differently when

A regulated obligation to retain full message content — some financial messaging regimes require it — would force capture, and it would then belong in a separate, encrypted, access-controlled store with its own retention and its own audit, not in the access-record stream.

Why it holds up over time

Data minimisation at the widest-reaching component is a durable rule. What changes is which component is widest-reaching this decade; the reasoning transfers to whatever replaces the gateway.