# Programmable Messaging API

*Programmable Messaging API · Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-10 · 22 views · 16 architecture decision records*

The argument these decisions serve is summarised in the [Architecture One-Pager](architecture-one-pager).

Sixteen decisions, grouped by the question they answer, each with the forcing question, the alternatives, what would flip the choice, and why it should outlast the technology it is realised on.

> **Status of this document.** This is a design, not a report on a running system. Every rate, latency, ratio, window and retention figure is a stated assumption, chosen to be defensible and arguable rather than measured. The operating context assumed throughout is a multi-tenant messaging platform serving 40,000 active tenants across 190 jurisdictions and 9 channels, with roughly 220 carrier and aggregator routes and 6,000 concurrent protocol sessions, accepting 8,000 messages per second at steady state with a 5x burst for 10 minutes, ingesting 25,000 delivery receipts and 2,500 inbound messages per second, and carrying about 20 billion outbound messages a month on Google Cloud in two regions. Where a carrier protocol or a channel's own rules settle a question, the protocol is followed and the records say so rather than claiming credit for it; every record here is about something the protocols deliberately leave to the platform.

## How to read a record

- **Question:** The forcing question: why a decision was needed at all.
- **Context:** The requirement, the scale and the constraint that make it hard.
- **Decision:** What this architecture does, stated so it can be checked.
- **How it is realised on the stack:** The concrete mechanism: which component or package, configured how, in which cluster.
- **Options weighed:** Chosen, rejected, deferred, or right elsewhere, with the reason for each.
- **Consequences:** What the choice buys and what it costs, both kept visible.
- **Choose differently when:** The conditions that would flip the decision for your system.
- **Why it holds up over time:** What keeps the decision right as scale, staff and technology change.
- **Lesson:** The principle that transfers beyond this platform.

## Decision map

**The ownership boundary**: The decision that defines the architecture, and the two that follow from it directly.

- ADR-01 · The accepted message is the unit of durability; the carrier attempt is disposable
- ADR-02 · Delivery state is a monotonic projection over attempts and receipts, not a stored field
- ADR-03 · Accept before route: submission never waits on a route, a session or a carrier

**Telling the truth about delivery**: What the platform does when the carrier is late, silent, contradictory, or talking about a message nobody recognises.

- ADR-04 · Unknown is a first-class terminal state, reported as such
- ADR-05 · The ambiguous submit is resolved by a recorded per-route policy, defaulting to no re-attempt
- ADR-06 · Receipts that correlate to nothing are quarantined, never discarded

**Capacity the platform rents**: Spending somebody else's throughput without losing the right to spend it, and deciding whose traffic waits.

- ADR-07 · Per-route throughput is enforced by leases checked out in blocks, not by a central broker or local guesswork
- ADR-08 · Three traffic classes with separate lanes, separate SLOs and a declared shedding order
- ADR-09 · Routing decides at attempt time, which makes the price quoted at submit an estimate

**Channels and sessions**: How much of a channel's strangeness is hidden, and where the one stateful thing in the platform lives.

- ADR-10 · A thin adapter contract, with per-channel capability declared rather than hidden
- ADR-11 · Carrier protocol sessions live in a dedicated stateful tier, not in the attempt fleet

**Consent, jurisdiction and evidence**: The rules the platform will be fined for, and the records that answer for them years later.

- ADR-12 · The platform owns regulated keywords; the tenant owns marketing preference
- ADR-13 · Jurisdictional rules are versioned data that fails closed, and changes without a deployment
- ADR-14 · Receipts and consent outlive the bodies they describe, and store-no-body is a supported mode

**Cost and operability**: What a message actually costs, and how a route earns the right to carry one.

- ADR-15 · One rated usage event per attempt, not per message
- ADR-16 · Route eligibility is earned from measured delivery, and absence of receipts is a signal

## Technology by capability

Every component on the twenty-two views, why it was chosen on Google Cloud, what would have been used instead, and the decision record that explains the trade. The topic belongs to no cloud; Google Cloud was chosen as the practice's honest rotation, and the requirement document stays vendor-neutral throughout.

| Capability | Choice | Origin | Credible alternative | Why this one | Record |
|---|---|---|---|---|---|
| Message ledger: append-only submissions and attempts, strongly consistent, RPO 0 | Cloud Spanner, multi-region instance, interleaved attempt rows under the message, partitioned by tenant and time | Google Cloud managed | A globally distributed SQL database with per-row region placement, or a partitioned relational cluster with synchronous replicas | Reading a message's whole history must be one local read, and the submit acknowledgement must wait on a committed replicated write; interleaving gives the first and synchronous replication gives the second without an application-level transaction | ADR-01 |
| Delivery-state projection: status by message id and by tenant client reference | Bigtable, keyed by message id with a secondary row keyed by tenant and client reference | Google Cloud managed | A wide-column store or a document store with a secondary index, rebuilt from the ledger either way | The projection is disposable and read-heavy at a rate the ledger should not serve; a key-value shape with predictable latency suits a fold result that is rebuilt rather than migrated | ADR-02 |
| Receipt and inbound streams, and the class lanes between submit and attempt | Pub/Sub topics: one per traffic class, one for receipts, one for inbound | Google Cloud managed | A managed Kafka or a durable log with per-partition ordering | The platform needs durability and independent scaling far more than it needs ordering, because the fold is order-independent by construction; separate topics give the class isolation that the shedding order depends on | ADR-08 |
| Per-route pacing against a contracted carrier window | Cloud Tasks, one queue per route with its dispatch rate set from the route table, beneath Memorystore leases | Google Cloud managed | A token-bucket service in front of every attempt, or per-worker budgets with periodic reconciliation | A managed queue with a configurable dispatch rate is a second, independent ceiling that survives a lease-store incident, which turns a potential contract breach into a throughput dip | ADR-07 |
| Route leases and route-quality counters | Memorystore for Redis, one instance per region, atomic block check-out by script | Google Cloud managed | A self-hosted in-memory store per region, or a consensus store if exactness were required | Single-key atomicity on one shard is exactly the guarantee a lease block needs and nothing more; the store is explicitly not authoritative, so its loss costs throughput rather than correctness | ADR-07 |
| Stateless submit, status, attempt execution and ingest workers | Cloud Run services, scaled independently per concern | Google Cloud managed | Containers on a managed Kubernetes cluster, which is what the session plane uses anyway | Four workloads with unrelated load curves need four scaling decisions, and none of them holds state; a request-scaled runtime makes the dormant-tenant cost floor near zero as the cost requirement demands | ADR-03 |
| Long-lived carrier protocol sessions with sequence windows | GKE StatefulSet with stable identities, one pod per bind, drained individually | Google Cloud managed Kubernetes | Dedicated virtual machines per carrier, or sessions co-located in the attempt workers | A bind is an addressable, individually recoverable thing with in-flight state; it needs stable identity and controlled termination, which is precisely what a request-scaled runtime cannot offer | ADR-11 |
| Carrier-approved static egress addresses | Cloud NAT with reserved addresses per region, one range per region allowlisted by each carrier | Google Cloud managed | A NAT appliance or per-carrier VPN and interconnect where a carrier requires private connectivity | Carriers allowlist source addresses by correspondence on their own timetable, so the platform needs a small, stable, reserved set per region rather than ephemeral egress | ADR-11 |
| Message bodies and media, with per-tenant keys and store-no-body mode | Cloud Storage with CMEK per tenant, lifecycle rules per retention setting | Google Cloud managed | An object store with customer-managed encryption and object lifecycle policies — the shape is the same anywhere | Content has a shorter and separately configurable life than the record of it, which requires a store that can be purged independently of the ledger | ADR-14 |
| Carrier credential custody, per-route scoping, tenant-managed keys | Secret Manager for route credentials with Cloud KMS for key material, reached only through a custody service | Google Cloud managed | A secrets manager behind a hardware security module where a tenant requires key custody in hardware | Attempt workers must hold no long-lived carrier material, so the exchange of workload identity for a short-lived single-route token needs a brokered boundary rather than mounted secrets | ADR-11 |
| Consent history, sender-identity registry, route table and jurisdictional rules | Cloud Spanner, append-only consent history and versioned configuration tables, cached in Memorystore with a staleness ceiling | Google Cloud managed | A relational cluster with synchronous replicas and an in-memory cache in front | Consent is read on every submit and must be strongly consistent; configuration is read a billion times and written rarely, so it belongs next to the reader with an explicit invalidation path | ADR-13 |
| Route quality aggregation and evidence analytics | Dataflow streaming jobs into BigQuery, feeding the Memorystore quality store | Google Cloud managed | A stream processor into a columnar warehouse — any equivalent pair | Delivery rate, receipt latency and unknown share are windowed aggregates over the receipt stream, and the same warehouse answers the regulator and the margin question | ADR-16 |
| Audit log retained independently of tenant settings | Cloud Logging sink into a Cloud Storage bucket with Bucket Lock, plus BigQuery for query | Google Cloud managed | An append-only object store with a retention lock and a columnar query layer | A seven-year audit obligation needs write-once retention the platform's own operators cannot shorten, which is a storage-level guarantee rather than an application one | ADR-14 |
| Edge: TLS, tenant credential check, WAF and anycast entry | Global external Application Load Balancer with Cloud Armor | Google Cloud managed | Any anycast load balancer with a managed WAF in front of the submit tier | Submit is a global API with a 99.99% target and hostile inbound endpoints beside it; anycast entry plus managed WAF keeps that off the application's critical path | ADR-03 |
| Build, provenance and deployment of services; separate path for rules and routes | Cloud Build with Artifact Registry and Binary Authorization; Terraform for infrastructure; a reviewed data-publication path for the route table and rule set | Google Cloud managed plus open-source Terraform | Any CI system with image signing and admission control, and any reviewed configuration pipeline | A jurisdictional rule change must take effect without a deployment, which means configuration needs its own reviewed path with the same scrutiny as code rather than a shortcut around the gates | ADR-13 |
| Hostile-receipt testing before staging | A carrier simulator suite as a blocking build gate: late, duplicated, out-of-order, silent and unknown-reference receipts | This design | Contract tests against provider sandboxes, which do not reproduce the pathologies that matter | Production cannot be asked to produce a four-hour-late duplicate receipt on demand, and those are precisely the inputs the fold and the quarantine exist for | ADR-02 |

## The decisions, and the alternatives that lost

### The ownership boundary

*The decision that defines the architecture, and the two that follow from it directly.*

#### ADR-01 · The accepted message is the unit of durability; the carrier attempt is disposable

**Status:** Accepted  ·  **Shown on views:** 02, 08, 11, 13

*A submitted message and a carrier call are both "the message" in ordinary speech. Which of them is the thing the platform durably owns, and what happens to the other one when it fails?*

**Context.** Every failure in this class of system looks different at first glance and identical underneath. A carrier returns 503. A carrier returns 429. A session's bind drops with forty messages in flight. A worker is descheduled mid-submit. A route is withdrawn by its provider. If the unit of durability is the carrier call, each of these is its own recovery path with its own state, and the platform ends up with five partially-correct mechanisms that disagree about what happened. If the unit is the accepted message, and the carrier call is a separate disposable record attached to it, all five become the same operation: append a fact about an attempt, and let the message's state follow. The cost is not small and has to be faced rather than discovered — durability on the submit path means the API cannot acknowledge until a replicated write has committed, and every attempt pays at least two more writes, which at 8,000 messages a second with retries and fallback hops is the hottest and most expensive path in the platform.

**Decision.** An accepted message is appended to a strongly consistent, append-only ledger before the API acknowledges, and that acknowledgement promises ownership, not delivery. Each carrier call is a separate append-only attempt record under that message, carrying its route, segment breakdown, provider reference, submit outcome and the receipts later correlated to it. Nothing durable lives in a worker or a session. No attempt is made without a stable attempt key, and no attempt record is ever updated in place.

**How it is realised on the stack.** The ledger is Cloud Spanner, partitioned by tenant and time so no single tenant's campaign creates a hot partition, with the message row and the attempt rows in the same interleaved family so reading a message's full history is one local read. Message bodies and media go to Cloud Storage under per-tenant CMEK keys with their own, shorter retention, so the ledger stays small and a body deletion does not touch the delivery record. Attempt execution runs on Cloud Run with no local state: an executor reads the message, appends intent, calls the carrier through the session plane or a provider's HTTP API, and appends the outcome. A descheduled executor loses nothing, because the intent record plus the absence of an outcome is itself a complete, resumable statement of what happened.

| Option | Verdict | Reasoning |
|---|---|---|
| Accepted message durable, attempt disposable | Chosen | Every carrier failure becomes one mechanism; pays two replicated writes per attempt and makes the ledger the platform's scaling constraint |
| The carrier call is the unit, retried from the start on failure | Rejected | Dramatically simpler and needs no ledger, but re-running a succeeded step means a second message to a human; for OTP traffic that is a wrong answer, not a slow one |
| Message durable, attempts kept only in a log for diagnostics | Rejected | Cheaper, and makes "what did we try" unanswerable at exactly the moment a tenant or a carrier disputes a charge |
| Queue-as-state: the message lives in the broker until delivered | Right elsewhere | Right for a platform with one channel, one route and no billing dispute to settle; here it cannot represent a fallback chain or two attempts at different prices |

**What it buys**

- Retry, park, re-route, session loss and worker death are one operation — appending a fact — rather than five recovery paths that can disagree
- A fallback chain is expressible without special handling: each hop is another attempt with its own cost and its own terminal state
- "What exactly happened to this message" is a single local read, which is what makes developer self-service diagnosis possible at all
- Every charge is attributable to an attempt, which is what makes a carrier billing dispute winnable

**What it costs**

- Two synchronously replicated writes per attempt make the ledger the platform's hottest write path and its primary scaling constraint
- The API cannot acknowledge faster than a replicated commit, which sets the floor under the submit latency budget
- Storage grows with attempts rather than messages, so a route with a poor acceptance rate costs more to record as well as to use

**Choose differently when.** If the platform only ever served one traffic class where a duplicate is harmless — server-to-server notifications with their own dedup, say — the ledger stops earning its cost and retry-from-the-start becomes the right answer. It would also flip if a carrier ecosystem emerged with universal, honoured idempotency keys: with the carrier able to answer "did you already take this?", the attempt record stops being the only way to know.

**Why it holds up over time.** The separation is between what was promised to a caller and what was tried on their behalf, and those have different lifetimes in any system that acts for somebody through a third party. That is true of a messaging platform, a payment orchestrator and a shipping integrator alike, and it stays true when the protocol underneath is replaced. The cost of the separation is paid per operation and the benefit grows with the number of distinct failure modes, which only ever increases as channels and providers are added.

> **Lesson.** When several failures look different but recover the same way, the unit of durability is probably drawn in the wrong place. Move it up until they collapse into one mechanism, then price what that costs on the hot path and say so out loud.

#### ADR-02 · Delivery state is a monotonic projection over attempts and receipts, not a stored field

**Status:** Accepted  ·  **Shown on views:** 02, 10, 12, 14

*Twenty-five thousand receipts a second arrive late, duplicated and out of order. Does the platform keep a status column and update it, or compute status from the facts?*

**Context.** A status column is the obvious design: fast to query, trivial to reason about, and one row per message. It is also a race the platform loses quietly. Carrier receipts are not ordered: a "delivered" generated at 10:00 can arrive after an "undelivered" generated at 09:58, from different parts of a carrier's estate, hours after the message's deadline, and sometimes twice. With a column, the last writer wins, which means the truth depends on network scheduling. Worse, a column needs a default for the case where nothing has arrived, and there is no honest default — writing "sent" implies more than is known and writing "failed" implies less. A fold over the attempt series and the correlated receipts has no such default: the fold of "one attempt accepted, no receipt, deadline passed" is unknown, and no code had to decide that.

**Decision.** The state model is accepted, queued, sent, then one of delivered, undelivered, expired or unknown, and it is computed as a monotonic fold over the message's attempts and the receipts correlated to them. Status is never written by the submit path. A transition out of a terminal state is forbidden, whatever a late receipt says. The served projection is a rebuildable read model, and the ledger plus the receipt stream remain the only authority.

**How it is realised on the stack.** A stream processor reads the receipt stream from Pub/Sub, correlates each receipt to an attempt by provider reference, appends it, recomputes the fold for that message and writes the result to a Bigtable projection keyed by message id, with a secondary row keyed by tenant and client reference so a tenant can query with their own identifier. The fold is a pure function of the appended facts, so replaying the stream reproduces the projection exactly. A TTL sweeper appends an explicit expiry fact when a deadline passes with no conclusive receipt, which is what makes unknown a derived value rather than an absence. Rebuilding the whole projection from the ledger is a batch job with a two-hour target, during which status queries are served from a lagging projection rather than failing.

| Option | Verdict | Reasoning |
|---|---|---|
| Status as a monotonic fold, served from a rebuildable projection | Chosen | Correct under any arrival order and has no dishonest default; status reads become reads of a cache that can lag and must be rebuildable |
| A mutable status column updated by each receipt | Rejected | Simplest and fastest, and silently corrupts state under out-of-order receipts, which are normal carrier behaviour rather than an edge case |
| A status column plus a timestamp guard, rejecting older receipts | Rejected | Fixes ordering using carrier clocks, which are not trustworthy across providers, and still needs a default for silence |
| Compute status on read, directly from the ledger, with no projection | Deferred | Removes a store and the rebuild problem; viable at a fraction of this read volume, and worth revisiting if status query rates stay low |

**What it buys**

- A receipt arriving four hours late, twice, or after the body has been deleted is a fact to append rather than a race to lose
- Unknown appears in the API, the dashboard, the SLO and the route-quality signal without anyone having chosen to put it there
- The projection is disposable, so a schema change to the read model is a rebuild rather than a migration
- Replay is exact, which makes a correlation bug correctable after the fact rather than permanent

**What it costs**

- Every status query reads a projection that can lag, and the lag has to be a monitored signal rather than an assumption
- Rebuilding at 20 billion messages a month must complete in hours, which constrains how expensive the fold may be
- Two stores hold overlapping information, and a reviewer will reasonably ask why

**Choose differently when.** If carriers converged on ordered, exactly-once receipt delivery with trustworthy timestamps, the ordering argument would weaken enough for a guarded column to be defensible. A collapse in read volume would also flip it: at low rates, computing from the ledger on read removes the projection entirely, which is why that option is deferred rather than rejected.

**Why it holds up over time.** The property that lasts is that arrival order is not under the platform's control and never will be, because the reporting parties are external and multiplying. A fold is indifferent to order; a column is not, and no amount of later engineering recovers information a lost write destroyed. The argument strengthens with every channel added, which is the opposite of how most simplifications age.

> **Lesson.** If a field has no honest default, it should not be a field. Compute it, and the uncomfortable answer appears by itself instead of being someone's decision to avoid.

#### ADR-03 · Accept before route: submission never waits on a route, a session or a carrier

**Status:** Accepted  ·  **Shown on views:** 02, 07, 13

*Should the submit call choose a route and confirm the carrier will take the message before acknowledging, so the tenant learns about problems immediately?*

**Context.** There is a real argument for routing at submit: the tenant finds out at once that no route exists, the price quoted is the price charged, and there is no queue to explain. The argument fails on availability arithmetic. Submit is required to hold 99.99% monthly; routes, sessions and carriers are third-party components with no such guarantee, and 220 of them. Coupling the submit path to any of them makes the platform's headline availability the product of everybody else's, and makes a single carrier's bad afternoon look like an outage of the API. It also inverts the latency relationship that matters: a tenant's HTTP call is in front of a user, and a carrier's acceptance is not.

**Decision.** Submission validates what is knowable without a carrier — destination, sender identity, jurisdictional sender rules, body limits, template approval, quota, credit and consent — appends the message, and acknowledges. Route selection, lease acquisition, adaptation and the carrier call all happen afterwards, asynchronously, and their failures are message states rather than API errors. The only synchronous external-ish dependency left on the submit path is the consent read, which is the platform's own store.

**How it is realised on the stack.** Cloud Run submit services behind a global load balancer with Cloud Armor; validation and sender-rule checks read a Memorystore-cached snapshot of the registry with a bounded staleness ceiling; consent is a strongly consistent Spanner read; the ledger append is the one durable write; the message is then published to a Pub/Sub lane topic for its traffic class. A dry-run endpoint gives tenants the thing they lose by this decision — resolved route, segment count, estimated price and compliance verdict — without sending anything, so the information is available when it is wanted rather than imposed on every call.

| Option | Verdict | Reasoning |
|---|---|---|
| Accept first, route later, with a dry-run for pre-flight answers | Chosen | Submit availability is independent of 220 third parties; the price returned at submit becomes an estimate and queueing must be explained to tenants |
| Route and confirm carrier acceptance synchronously | Rejected | Truthful pricing and immediate errors, bought by making the API's availability the product of every carrier's and its latency the carrier's latency |
| Hybrid: synchronous for interactive, asynchronous for bulk | Rejected | Two submit semantics in one API, and the class that most needs availability would be the one coupled to carriers |
| Synchronous route selection without carrier confirmation | Deferred | A middle position that makes the quoted price firmer at modest cost; blocked for now by ADR-09, which wants route health evaluated at attempt time |

**What it buys**

- The submit SLO is achievable without any route, session or carrier being healthy
- A carrier outage is a queue and a set of message states, not an API error rate
- Load from a four-million-message campaign lands on the scheduler, not on the API's latency profile
- Validation rejections stay fast and specific, because they need nothing external

**What it costs**

- The segment count and price returned at submit are estimates, and a re-route or fallback hop can exceed them
- "Accepted" has to be explained, and some tenants will read it as "delivered" however it is documented
- A message can be accepted and then fail for a reason that was knowable at submit time, such as a route having just been withdrawn

**Choose differently when.** A single-channel, single-route platform with one carrier contract and no fallback could afford synchronous confirmation, because there is nothing to be independent of. It would also flip for a product whose tenants genuinely cannot tolerate an estimated price — a wholesale reseller, say — where firm pricing at submit is worth coupling to.

**Why it holds up over time.** Separating the promise to accept from the attempt to deliver is the oldest pattern in integration and the one that keeps a platform's availability its own. What changes over a decade is the number of third parties, which only grows; the value of not being coupled to them grows with it.

> **Lesson.** Before coupling a path to a dependency, multiply the availability targets. If the product is below the path's own SLO, the coupling is the decision being made, whatever else the design discussion is about.

### Telling the truth about delivery

*What the platform does when the carrier is late, silent, contradictory, or talking about a message nobody recognises.*

#### ADR-04 · Unknown is a first-class terminal state, reported as such

**Status:** Accepted  ·  **Shown on views:** 12, 14, 19

*A message was accepted by a carrier, no receipt ever arrived, and the deadline has passed. What does the platform tell the tenant?*

**Context.** Some routes never report. Some report for 90% of traffic. Some report only failures. A platform that insists on a binary answer must therefore invent one, and the invented answer is always the optimistic one, because the sender wants to believe it and nobody downstream can contradict it. The consequences compound: an inflated delivery rate becomes the route-quality signal that drives routing, so the worst-reporting routes look the best; the SLO measures a number the platform made up; and the developer whose customer did not receive a code is told the message was delivered, which converts a solvable support question into a dispute about the platform's honesty.

**Decision.** Unknown is a terminal state in the model, reached when a message's time-to-live elapses with no conclusive receipt. It is returned by the API, shown in the console, counted in the SLOs, reported per route, and used as a route-quality input. It is never collapsed into delivered or undelivered, and the absence of a receipt is never treated as success.

**How it is realised on the stack.** A TTL sweeper appends an explicit expiry fact to the ledger when a deadline passes with no conclusive receipt, so unknown is derived by the same fold as every other state rather than inferred by a query. Unknown share is aggregated per route in BigQuery and fed back into the Memorystore route-quality store, so a route that stops reporting loses weight in the least-cost and best-quality policies. The tenant-facing vocabulary distinguishes three things explicitly: delivered, undelivered with a reason, and unknown with the attempt trail that produced it.

| Option | Verdict | Reasoning |
|---|---|---|
| Unknown as a terminal state, reported and used as a signal | Chosen | Honest, and produces a worse-looking delivery rate than competitors who round; requires the distinction to be explained in the product |
| Treat no-receipt as delivered after the deadline | Rejected | What the sender wants to hear, and it poisons the routing signal, the SLO and the support conversation at the same time |
| Treat no-receipt as undelivered after the deadline | Rejected | Pessimistic in the other direction: triggers tenant retries for messages that did arrive, which means duplicates on handsets |
| Per-route configuration of the assumed outcome | Rejected | Makes the platform's reported truth a configuration value, which is the same lie with an audit trail |

**What it buys**

- Routing is driven by observed delivery rather than by an assumption that flatters bad reporters
- The developer's "nothing arrived" conversation starts from a platform answer rather than a platform denial
- The SLO measures something real, which means it can be improved rather than defended
- Carriers with poor reporting become visible as a commercial problem rather than a silent quality tax

**What it costs**

- The headline delivery rate is lower than a competitor's rounded figure, and sales will have to argue the point
- Unknown share varies by route for reasons outside the platform's control and will be read as a platform defect
- Tenants need a documented position on what to do with an unknown, which is a product decision as much as an API one

**Choose differently when.** Nothing short of universal, reliable carrier reporting would flip this, and that is a commercial change rather than a technical one. If a jurisdiction's regulator mandated receipt reporting and it were enforced, unknown would shrink to a rounding error there and the state could in principle be retired for those routes — but the model would keep it, because the first new channel would need it again.

**Why it holds up over time.** Any platform that acts through parties it cannot observe will have outcomes it cannot determine, and the share of them grows as the number of parties grows. Naming that category is a stance that does not depend on the technology; abandoning it is irreversible, because once a comfortable default exists every downstream consumer inherits it and the distinction cannot be recovered.

> **Lesson.** A system should be allowed to say it does not know. Removing that option does not create knowledge — it relocates the guess to the place where it is least visible and most expensive.

#### ADR-05 · The ambiguous submit is resolved by a recorded per-route policy, defaulting to no re-attempt

**Status:** Accepted  ·  **Shown on views:** 13, 16, 19

*An attempt times out with no response and the carrier offers no idempotency key. The message may have been delivered, dropped, or never received. Does the platform try again?*

**Context.** This is the sharpest trade in the platform and it has no answer that is right everywhere. Re-attempting risks a second message to a person — two login codes, two appointment reminders — and a second charge the carrier will also levy. Not re-attempting risks a silent non-delivery that the tenant will attribute to the platform. Probing is only available where a route supports querying a provider reference, which most do not. The right answer genuinely differs: for an OTP, a duplicate is confusing but a missing code blocks a login; for a marketing message, a duplicate is a complaint and a miss is nothing. A single platform-wide rule will therefore be wrong for a large share of traffic whichever way it is set.

**Decision.** An attempt that times out with no response is recorded as an ambiguous outcome, which is a fact rather than a failure. Whether to re-attempt is a per-route policy with a per-traffic-class override, and the policy applied is written onto the message so the decision is auditable. The default is not to re-attempt on a route that cannot be queried. Where a route supports querying by provider reference, the policy is to probe before deciding.

**How it is realised on the stack.** The session plane records the ambiguity when a sequence window times out, and the attempt's outcome field carries it explicitly rather than being left empty. The route table holds three fields per route — queryable, default ambiguity policy, and per-class overrides — as versioned control-plane data, so the policy can change without a deployment. The executor reads the policy, acts, and appends the policy version it applied. Ambiguous-submit rate is a reported metric per route, and the measured duplicate rate of re-attempting routes is published beside the unknown rate of the others, so the trade is argued with numbers rather than with intuition.

| Option | Verdict | Reasoning |
|---|---|---|
| Per-route policy with class overrides, default no re-attempt | Chosen | Protects the recipient by default and lets a defensible tenant choose otherwise; needs the policy recorded per message or the decision is unauditable |
| Always re-attempt | Rejected | Maximises apparent delivery and puts duplicate codes on handsets and duplicate charges on invoices |
| Never re-attempt, no override | Rejected | Simple and defensible, and wrong for interactive traffic where a missing code is worse than a duplicate |
| Always probe | Right elsewhere | The correct answer on a route estate where querying is universal; here most routes cannot be queried, so it would be a policy that mostly does not apply |

**What it buys**

- The default protects the person holding the handset, which is the actor with no relationship to the platform
- The trade is visible and arguable per route, with measured duplicate and unknown rates beside it
- Every message carries the policy that decided its fate, so a dispute three weeks later has an answer
- Routes that support querying get a materially better outcome, which gives carriers a reason to offer it

**What it costs**

- Two tenants on the same channel can legitimately see different behaviour, which is harder to document than one rule
- The default accepts silent non-deliveries, and some of them will be login codes
- Policy is control-plane data that changes behaviour, so it needs the same review as code

**Choose differently when.** Universal honoured idempotency keys across carrier protocols would remove the ambiguity and the decision with it. Short of that, measured evidence could flip the default: if the duplicate rate from re-attempting turns out to be far lower than the unknown rate it prevents, the interactive default should change — which is exactly what the prototype is asked to measure.

**Why it holds up over time.** Ambiguity under timeout is inherent to any remote call without an idempotency mechanism, and will outlive every protocol in this design. What lasts is the shape of the answer: make the ambiguity a recorded fact, make the resolution a policy rather than a constant, and record which policy ran. That remains correct whether the right default moves or not.

> **Lesson.** When a trade-off genuinely differs by context, the architecture's job is to make the choice explicit, per context, and recorded — not to pick a side and hard-code it where nobody can see it.

#### ADR-06 · Receipts that correlate to nothing are quarantined, never discarded

**Status:** Accepted  ·  **Shown on views:** 12, 14, 19

*A receipt arrives carrying a provider reference the platform has no record of. Is it noise to drop, or data to keep?*

**Context.** The instinct is to drop it: an unknown reference looks like a carrier's mistake, a test from their side, or a replay. In practice it is usually the platform's own fault — a correlation key written wrongly, an attempt record that failed to commit, a provider that changed its reference format, a route migrated between accounts — and each of those is a defect that the dropped receipts were the only evidence of. The remainder matter commercially: a carrier invoice lists messages the platform cannot match, and without the receipts there is nothing to dispute with. Dropping is also irreversible in a way that keeping is not.

**Decision.** An uncorrelatable receipt is appended to the receipt stream and written to a quarantine store with its full payload, its route, and the reason correlation failed. It is never discarded. The quarantine has an owner, an age distribution and an alert on its rate of change, and a quarantined receipt may age out only after a named cause has been recorded. The data model permits this directly: the receipt's attempt reference is nullable by design.

**How it is realised on the stack.** Correlation runs on Cloud Run workers reading the receipt stream; a failed match writes to a Bigtable quarantine table partitioned by route and day, retained 30 days, with the raw payload preserved verbatim. Rate of change rather than level is alerted, because a steady background trickle from one provider is a known condition and a step change is a new defect. Quarantined receipts are replayable: once a correlation bug is fixed, the window can be re-processed and the folds recomputed, because state is a projection rather than a stored field.

| Option | Verdict | Reasoning |
|---|---|---|
| Quarantine with full payload, owner and rate alerting | Chosen | Preserves the evidence of the platform's own defects and of billing disputes; costs a store and, more importantly, somebody's attention |
| Drop with a counter | Rejected | Cheap, and it converts the earliest signal of a correlation defect into a number nobody can act on |
| Log the payload and rely on log search | Rejected | Looks equivalent and is not: logs are retained on a different clock, are not replayable into the fold, and are rarely retained for the 13 months a dispute needs |
| Reject at the endpoint with an error to the carrier | Rejected | Teaches carriers to retry or to stop sending, and loses the payload either way |

**What it buys**

- Correlation defects are discoverable and, because state is a projection, correctable after the fact by replay
- A carrier invoice line the platform cannot match has evidence behind it
- A provider changing its reference format shows up as a quarantine step change rather than as a silent drop in delivery rate
- The nullable reference in the model makes the intent explicit to anyone reading the schema

**What it costs**

- A store, a retention policy and a named owner for data that is, by definition, not useful yet
- A quarantine with no owner becomes a dashboard nobody reads, which is worse than a counter because it looks like diligence
- Raw payloads may carry content, so the quarantine needs the same classification and encryption as the body store

**Choose differently when.** If the quarantine rate stayed at zero for a long period across all routes, the store could be reduced to sampling. It would also change if receipts were self-describing and signed in a way that let the platform prove a reference was never issued by it — then an unmatched receipt really would be somebody else's problem.

**Why it holds up over time.** Any system correlating externally generated events to internal records will have events it cannot place, and the rate of them is a direct measure of its own correctness. Keeping them is how the system learns about itself. That is independent of protocol, channel and decade, and the cost of keeping them stays small relative to what they reveal.

> **Lesson.** Data a system cannot explain is the most valuable data it has, because it is a measurement of the system's own understanding. Discarding it is discarding the only instrument pointed inward.

### Capacity the platform rents

*Spending somebody else's throughput without losing the right to spend it, and deciding whose traffic waits.*

#### ADR-07 · Per-route throughput is enforced by leases checked out in blocks, not by a central broker or local guesswork

**Status:** Accepted  ·  **Shown on views:** 07, 15, 17, 20

*A carrier's window of 300 messages per second is contractual, and breaching it suspends a sender identity. How is that enforced across a fleet that scales from two workers to forty?*

**Context.** This is not a rate limit in the ordinary sense. Exceeding an API's quota earns a 429; exceeding a carrier's contracted window earns a throttled or suspended sender identity, a commercial conversation, and sometimes a jurisdiction the platform cannot send into for a week. So the enforcement has to hold across an elastic fleet, and it has to hold while sitting inside an interactive latency budget of two seconds end to end. A central broker every attempt passes through is correct and puts a network hop, and a new single point of failure for all sending, on the hot path. Per-worker local budgets reconciled periodically are fast and overshoot by exactly whatever the reconciliation window allows, which is the wrong variable to be imprecise about. Leases checked out in blocks are the compromise, and their failure mode is a crashed worker holding capacity nobody can spend.

**Decision.** Each route's window is divided into leases. A worker checks out a block of permits for a route, spends them locally with no further coordination, and returns or forfeits the remainder. Block size and lease duration are per-route configuration, sized so that the worst-case stranded capacity is a stated fraction of the window rather than an unknown. Lease state is regional and explicitly not authoritative for correctness: losing it costs throughput, never a duplicate or a lost message.

**How it is realised on the stack.** Leases live in Memorystore for Redis, one instance per region, with block check-out as a single atomic script so a lease is never double-issued. Cloud Tasks queues provide the pacing underneath, one queue per route with its dispatch rate set from the route table, which gives a second, independent ceiling that survives a lease-store incident. Leases are deliberately regional rather than shared across regions: a shared lease store would put a cross-region round trip inside the interactive budget, and would make a carrier window depend on inter-region connectivity. The split of a route's window between regions is configuration, reviewed like any other route data.

| Option | Verdict | Reasoning |
|---|---|---|
| Leases checked out in blocks, regional, with a pacing queue underneath | Chosen | No hop per attempt and a bounded, stated overshoot; a crashed worker strands a block until its lease expires |
| Central lease broker on every attempt | Rejected | Exact enforcement, bought with a hop inside the interactive budget and a component whose outage stops all sending |
| Per-worker local budgets with periodic reconciliation | Rejected | Fastest, and the overshoot is whatever the window allows — unacceptable when the penalty is suspension rather than a 429 |
| One sending worker per route, so the window needs no coordination | Right elsewhere | Right for a platform with a handful of routes; at 220 routes it is 220 singleton workloads and a availability problem per route |

**What it buys**

- Attempt execution pays no coordination cost per message, which is what makes the interactive p95 reachable
- Worst-case overshoot is a configured number rather than an emergent one, so it can be negotiated with a carrier
- The pacing queue is a second independent ceiling, so a lease-store incident degrades throughput rather than breaching a contract
- Regional leases keep a carrier window independent of inter-region connectivity

**What it costs**

- A worker that dies holding a block strands that capacity until the lease expires, which shows as a throughput dip
- Block size and lease duration are per-route tuning parameters, and getting them wrong is either overshoot or waste
- A route's window must be split across regions in configuration, which is one more thing to get wrong during a regional event

**Choose differently when.** If a carrier offered a window enforced on their side with a clean, cheap backpressure signal rather than a penalty, local budgets would be sufficient and the lease machinery could go. Conversely, if sub-millisecond coordination became available in-region, a central broker's hop would stop mattering and exactness would be worth taking.

**Why it holds up over time.** Platforms will always be spending capacity they rent under terms they did not set, and the gap between an elastic fleet and a fixed external window will not close. Leasing a divisible right to spend, locally, with a bounded loss on failure, is the general answer; only the store and the latency numbers change.

> **Lesson.** Match the strength of a coordination mechanism to the cost of being wrong, not to the elegance of being exact. When the penalty is commercial rather than technical, a stated worst case beats an unbounded approximation and a hop you cannot afford.

#### ADR-08 · Three traffic classes with separate lanes, separate SLOs and a declared shedding order

**Status:** Accepted  ·  **Shown on views:** 07, 15, 19

*A tenant submits four million marketing messages while another tenant sends one login code. How does the platform decide what waits?*

**Context.** Interactive and bulk traffic are different products that happen to share an API. A login code is worthless thirty seconds late; a campaign is fine arriving over four hours, and its value is in completing inside a window rather than in any individual message's latency. One queue with a priority field averages the two: the campaign's volume raises the code's queue age, and the code's urgency distorts the campaign's pacing. Under overload, a single queue has only one lever — everything slows — which is the worst available outcome, because it degrades the traffic that cannot tolerate degradation in order to protect traffic that could have waited.

**Decision.** Every submission declares a traffic class: interactive, conversational or bulk. The classes get separate queues, separate scheduling, separate route preferences and separate SLOs. Scheduling is by class first and fairness within class second, with no tenant taking more than 40% of a shared route's capacity while another tenant has work queued for it. Under overload the platform sheds by refusing in a declared order — bulk, then conversational, then interactive — rather than by extending latency across the board.

**How it is realised on the stack.** Separate Pub/Sub topics per class, with subscriber fleets scaled independently, so bulk backlog cannot occupy interactive consumer capacity. Bulk carries a route preference biased to cost and is paced to the tenant's stated completion window; interactive is biased to measured quality and gets first claim on lease blocks. Queue age is reported per class and per route and exposed to tenants, so a paced campaign is visibly paced rather than apparently stuck — which is the support cost this decision creates and must therefore answer. Class is required at submit rather than inferred, because inference from content or volume is wrong exactly when it matters.

| Option | Verdict | Reasoning |
|---|---|---|
| Three declared classes, separate lanes, declared shedding order | Chosen | Protects the traffic that cannot wait and makes overload behaviour predictable; tenants must classify honestly and pacing must be made visible |
| One queue with a priority field | Rejected | Simplest, and under load it degrades everything to protect nothing in particular |
| Infer the class from volume and content | Rejected | Removes a field from the API and gets the answer wrong for the single urgent message inside a bulk submission |
| Per-tenant dedicated capacity instead of classes | Right elsewhere | Right for a small number of large tenants; at 40,000 tenants the cost floor per tenant makes it uneconomic |

**What it buys**

- Interactive queue age holds regardless of concurrent bulk volume, which is the platform's core latency promise
- Overload behaviour is a stated order of refusal rather than an emergent slowdown
- Bulk can use cheaper routes and longer windows without compromising the interactive quality bar
- Fairness is expressible per route, which is where contention actually happens

**What it costs**

- Tenants can misclassify, deliberately or carelessly, and entitlement has to police it
- Three lanes is three sets of capacity decisions and three SLOs to measure and defend
- Pacing must be made visible, or every paced campaign becomes a support ticket about being stuck

**Choose differently when.** A platform serving only transactional traffic would not need the split, and one serving only campaigns would not either. Classes earn their complexity precisely because both share the ledger. If bulk were ever moved to a separate product with its own API, this decision would collapse back to a single lane.

**Why it holds up over time.** Mixing workloads with different value-of-latency on shared scarce capacity is a permanent condition of multi-tenant platforms. Declaring the classes and the shedding order is what makes behaviour under stress a design property rather than a discovery, and that reasoning survives any change in the queueing technology.

> **Lesson.** Decide what to refuse before you are forced to. A system without a declared shedding order has one by default, and it is usually the order that protects the least important thing.

#### ADR-09 · Routing decides at attempt time, which makes the price quoted at submit an estimate

**Status:** Accepted  ·  **Shown on views:** 05, 09, 20

*Is the route chosen when the message is accepted, or when the attempt actually runs?*

**Context.** Choosing at submit has real virtues: the price quoted at acceptance is the price charged, the decision is auditable against exactly one version of the route table, and a support engineer three weeks later has an unambiguous answer. It also binds a message to a route that may be withdrawn, throttled or silently failing by the time the attempt runs — which, for a bulk campaign paced over four hours, is not a corner case but the normal condition. Choosing at attempt time uses current health and current quality, and pays by making the submit-time price an estimate that a re-route or a fallback hop can exceed. The two are not equally wrong: a stale route produces a failed or unknown message, while an estimated price produces a conversation.

**Decision.** Route selection happens per attempt, against the route table version current at that moment, under a tenant-visible policy of least cost, best measured quality, or an explicitly pinned route. The attempt record carries the route chosen, the policy that chose it, the table version, and the alternatives considered. The submit response returns an estimated segment count and price, labelled as an estimate, and the dry-run endpoint gives a firm pre-flight answer for tenants who need one before sending.

**How it is realised on the stack.** The route table and prices live in Spanner and are cached in Memorystore with an explicit invalidation path, so a selection costs a local read. The selector reads the Memorystore quality store — delivery rate, receipt latency and unknown share per route, recomputed continuously from the receipt stream — and applies the tenant's policy. The alternatives considered are written onto the attempt as a short list with their scores, because a routing decision nobody can reconstruct is indistinguishable from a random one when a tenant asks why their message cost three times the quote.

| Option | Verdict | Reasoning |
|---|---|---|
| Select per attempt, record the decision and the alternatives | Chosen | Uses current health, so a withdrawn route never strands a queued message; the submit price becomes an estimate and must be labelled as one |
| Select at submit and bind the message to the route | Rejected | Firm pricing and a single auditable table version, bought by sending queued messages down routes that have since died |
| Select at submit, re-select only on failure | Rejected | A reasonable middle that still binds the first attempt to stale health, which is the attempt that matters for interactive traffic |
| Let tenants pin a route always | Right elsewhere | Right for a wholesale customer with their own carrier contracts; offered as a policy, not made the default |

**What it buys**

- A route withdrawn or collapsed after submit never receives the queued traffic, which matters most for long-paced campaigns
- Quality signals have somewhere to act, which is what makes the route lifecycle loop meaningful rather than decorative
- Every routing decision is reconstructible from the attempt record, including what was not chosen
- Interactive failover needs no operator, because the next attempt simply reselects

**What it costs**

- The price returned at submit is an estimate, and a fallback or re-route can exceed it
- Tenants can be charged different amounts for identical messages minutes apart, which needs explaining in the product
- Storing alternatives considered on every attempt adds to the platform's hottest write path

**Choose differently when.** If route health were stable over hours rather than minutes — a small, reliable route estate under direct contract — selecting at submit would be the better trade, because firm pricing has real commercial value and the staleness risk would be negligible. A regulatory requirement to quote a firm price before sending would also flip it.

**Why it holds up over time.** Deciding late is correct whenever the information needed to decide well decays faster than the work waits. That relationship is a property of the dependency, not of the technology, and it holds for carrier routes, for capacity markets and for any supplier whose availability is measured in minutes.

> **Lesson.** Bind to a supplier as late as the product's promises allow. Early binding buys certainty about price and spends it on certainty about availability, and only one of those two is usually recoverable afterwards.

### Channels and sessions

*How much of a channel's strangeness is hidden, and where the one stateful thing in the platform lives.*

#### ADR-10 · A thin adapter contract, with per-channel capability declared rather than hidden

**Status:** Accepted  ·  **Shown on views:** 09, 15

*How much of a channel's strangeness should the platform absorb — template approval, rich cards, media rules, conversation windows — and how much should the API expose?*

**Context.** A thick adapter promises one message that works everywhere: the platform negotiates capability, degrades a rich card to text, substitutes an approved template, and the tenant sends the same payload to every channel. It is attractive until the degradation matters. A marketing message designed as a carousel, silently delivered as 160 characters, is a different product than the one the tenant authored, and the platform has quietly taken responsibility for an editorial decision. A thin adapter keeps a small common core — destination, sender identity, body or template reference, media reference — and makes the per-channel requirements explicit, visible fields that a tenant must satisfy. It leaks the carriers' inconsistencies into the API, which is honest and more work for the tenant.

**Decision.** The internal adapter contract is deliberately thin: submit an attempt, receive an acceptance or rejection, receive asynchronous receipts. Channel-specific requirements are first-class, declared, tenant-visible properties — template approval state, media constraints, conversation windows, capability probe results — and a fallback is an explicit tenant-declared chain where each hop is a separate attempt with its own cost and terminal state, never an invisible degradation.

**How it is realised on the stack.** Each channel is a Cloud Run adapter implementing the same three-method contract plus a capability declaration and a retryability map for its error codes. The message spine knows only messages, attempts and routes; adding a channel adds an adapter and a route family and touches neither the ledger schema nor the state model, which is the property the thin contract exists to preserve. Where a channel needs a probe — RCS capability, for instance — the probe result is recorded on the attempt, so a fallback to SMS is explicable rather than mysterious, and both hops appear in the rated usage.

| Option | Verdict | Reasoning |
|---|---|---|
| Thin contract, declared capability, explicit fallback chains | Chosen | No silent degradation and no change to the spine per channel; tenants carry per-channel knowledge the platform could have hidden |
| Thick adapter with automatic capability negotiation | Rejected | A simpler API, and it makes the platform responsible for editorial decisions nobody asked it to make |
| Lowest-common-denominator message only | Rejected | Thinnest of all, and it makes rich channels unusable, which is most of the roadmap |
| Per-channel APIs with no common core | Right elsewhere | Right for a platform selling one channel deeply; here it forfeits the fallback chain and multiplies the tenant's integration work |

**What it buys**

- Adding a channel is an adapter and a route family, with no change to the ledger, the state model or the spine
- A fallback is two attempts with two prices and two outcomes, which is both honest and billable
- A tenant's rich message is never silently reduced without the fallback being visible in the attempt trail
- The retryability map per channel is explicit, so a new channel's errors are classified rather than guessed

**What it costs**

- Tenants must learn per-channel rules the platform could have hidden, which raises the integration cost the developer journey measures
- Capability probes add latency and a dependency on a channel's own availability before the real send
- Multi-channel sends need the tenant to author per channel, or to accept a plainer message everywhere

**Choose differently when.** If the roadmap committed to many channels for tenants who genuinely author once and send everywhere — a notification product rather than a messaging API — the thick adapter becomes right, and the editorial responsibility is one the product should take. The boundary should be re-examined the moment a tenant segment appears that never authors per channel.

**Why it holds up over time.** Channels will keep diverging, because their differences are commercial rather than technical. A thin boundary ages better than a translation layer, because a translation layer has to be updated every time a channel changes and is wrong in the interval; a declared capability is merely incomplete.

> **Lesson.** Hiding a difference is taking responsibility for it. That is sometimes right, but it should be a decision with a named owner rather than a convenience that arrives with the abstraction.

#### ADR-11 · Carrier protocol sessions live in a dedicated stateful tier, not in the attempt fleet

**Status:** Accepted  ·  **Shown on views:** 08, 15, 17, 21

*SMPP binds are long-lived TCP sessions with sequence windows and allowlisted source addresses. Do they live inside the attempt workers, or in a tier of their own?*

**Context.** Co-locating binds with the attempt workers removes a network hop from the hot path and keeps the deployment simpler. It also makes every autoscaling event a protocol event: a scale-in tears down a bind with messages in flight, each of which becomes an ambiguous submit — the single most expensive outcome in the platform. It makes the carrier's source-address allowlist a function of the fleet's ephemeral egress, and it couples the attempt fleet's scaling policy to how many binds a carrier permits, which is usually a small number fixed by contract. A dedicated tier costs a hop and an extra deployable, and gives each bind a stable identity, controlled termination and an egress address a carrier has approved.

**Decision.** Carrier protocol sessions run in a dedicated stateful tier with stable per-bind identity, individual recovery, controlled draining, and reserved egress addresses allowlisted per region. The attempt fleet calls that tier over an internal contract and scales freely. Session loss, re-bind storms and sequence-window exhaustion are treated as ordinary operating events with defined handling, not incidents.

**How it is realised on the stack.** A GKE StatefulSet gives each bind a stable identity and a termination hook that drains in-flight attempts before closing, so a deliberate scale-down produces no ambiguity. Egress leaves through Cloud NAT with reserved addresses, one small range per region, allowlisted by each carrier in advance — which makes a second region a correspondence exercise with each carrier, tracked as a per-route dependency with its own state. Re-binds are capped and jittered to avoid a thundering herd after a carrier restart. In-flight attempts at the moment of an uncontrolled session loss are resolved as ambiguous under ADR-05, rather than silently retried.

| Option | Verdict | Reasoning |
|---|---|---|
| Dedicated stateful session tier with reserved egress | Chosen | Scaling the attempt fleet stops being a protocol event; costs one internal hop and a second deployable with its own scaling axis |
| Binds co-located in the attempt workers | Rejected | One less hop, and every scale-in becomes a teardown with in-flight ambiguity and an egress address the carrier has not approved |
| Dedicated virtual machines per carrier | Deferred | Maximum isolation and stable addressing; worth revisiting for a small number of very large carriers where per-bind tuning matters |
| Delegate all carrier connectivity to aggregators over HTTP | Right elsewhere | Right for a platform that never wants protocol work; forfeits direct routes, which is where margin and quality control live |

**What it buys**

- The attempt fleet autoscales without creating ambiguous submits, which protects the platform's most expensive failure class
- Each bind can be drained, restarted and tuned individually without affecting the others
- Egress addresses are a small, stable, pre-approved set per region rather than a function of fleet size
- Carrier-permitted bind counts stop constraining the attempt fleet's scaling policy

**What it costs**

- An internal hop inside the interactive latency budget, and a second deployable with its own failure modes
- A stateful workload in a platform that is otherwise entirely stateless, with the operational knowledge that implies
- Regional failover depends on carriers approving a second address range on their timetable, not the platform's

**Choose differently when.** If the route estate moved entirely to HTTP aggregators, the session tier would have nothing to hold and should be removed rather than maintained. Conversely, if direct carrier relationships grew to dominate margin, the deferred per-carrier virtual machine option becomes attractive for the largest few.

**Why it holds up over time.** Stateful protocol endpoints and elastic stateless fleets have incompatible lifecycles, and that incompatibility does not improve with better orchestration. Isolating the stateful part behind an internal contract is the general answer, and it keeps the blast radius of the strange component small while the rest of the platform evolves freely.

> **Lesson.** When one component's lifecycle fights the platform's, give it a tier rather than a workaround. The hop is cheaper than letting every scaling event become a correctness question.

### Consent, jurisdiction and evidence

*The rules the platform will be fined for, and the records that answer for them years later.*

#### ADR-12 · The platform owns regulated keywords; the tenant owns marketing preference

**Status:** Accepted  ·  **Shown on views:** 06, 15, 21, 22

*Somebody texts STOP to a short code four tenants share. Whose job is it to make the messages stop?*

**Context.** If consent belongs entirely to the tenant, the platform is a courier that cannot enforce the one rule it will be fined for, and an inbound STOP has nowhere authoritative to land: it is forwarded to whichever tenant the platform guesses, and the other three keep sending. If consent belongs entirely to the platform, it takes on liability for every tenant's marketing preferences and puts a strongly consistent store on the hot path of every submit. The split is obvious and is its own commitment: two consent systems that can disagree about the same destination, and a boundary that has to be explained to every tenant and every regulator.

**Decision.** The platform owns the regulated keywords and the consent evidence. An inbound STOP, UNSUBSCRIBE or locale equivalent is intercepted before the tenant is notified: the consent change is written to an append-only history, applied immediately, the mandated confirmation is sent, and the message is forwarded to the tenant as an already-actioned event. Tenants own marketing preference and may suppress more, never less. Consent is evaluated on the submit path for every message in a class that requires it, and that evaluation fails closed.

**How it is realised on the stack.** Inbound endpoints verify and deduplicate by route and provider reference, then the keyword interceptor writes a consent event to Spanner before anything is dispatched to the tenant. Consent events are append-only and queryable by destination, so "what did we know when we sent this" is answerable years later, and the message's refusal carries the consent record version consulted. Shared sender identities resolve tenant ownership by an explicit allocation rule rather than by conversation history. Propagation to the data plane is bounded at 30 seconds and published as a commitment, which makes messages already accepted and in flight a reported number rather than an unexamined gap.

| Option | Verdict | Reasoning |
|---|---|---|
| Platform owns regulated keywords; tenant may suppress more | Chosen | The rule the platform is fined for is the rule it enforces; costs a strongly consistent read on every submit and a boundary to explain |
| Tenant owns all consent; platform forwards keywords | Rejected | Lower liability on paper, and a STOP on a shared number silences at most one of four tenants |
| Platform owns all consent including marketing preference | Rejected | Eliminates disagreement and makes the platform the arbiter of every tenant's marketing policy, which it is not equipped to be |
| Per-tenant choice of who owns consent | Rejected | A configurable regulatory boundary is a boundary that will be configured wrongly exactly once |

**What it buys**

- A STOP on a shared sender identity is honoured across every tenant using it, immediately
- Consent evidence is append-only and queryable per destination, which is what a regulator or a complaint actually needs
- A refusal is attributable: machine-readable reason plus the rule and consent versions consulted
- Tenants cannot accidentally under-suppress, because the floor is the platform's

**What it costs**

- A strongly consistent consent read sits inside the submit latency budget on every message in a consent-bearing class
- Consent evaluation must hold 99.99%, making it one of the platform's availability ceilings
- Two consent systems exist, and the boundary between them has to be documented for tenants and regulators alike

**Choose differently when.** A platform with strictly dedicated sender identities per tenant could leave consent with the tenant, because the shared-number problem disappears and liability follows the identity. It would also change in a jurisdiction that made the sender legally responsible in a way the platform could evidence without holding the record.

**Why it holds up over time.** Where liability sits is a legal fact, not an engineering preference, and the rule is simple and lasting: the party that will be fined must be the party that can enforce. Shared identities and multi-tenancy are not going away, so neither is the need for a platform-owned floor beneath tenant preference.

> **Lesson.** Put enforcement where the liability is. A platform that can be fined for something it cannot prevent has drawn a boundary for its own convenience rather than for the obligation.

#### ADR-13 · Jurisdictional rules are versioned data that fails closed, and changes without a deployment

**Status:** Accepted  ·  **Shown on views:** 13, 18, 21

*Sender-ID registration requirements, quiet hours, template pre-approval and frequency caps differ across 190 jurisdictions and change without notice. Is that code or data?*

**Context.** As code, every regulatory change is a release: a pull request, a pipeline, a deploy window, and an interval during which the platform is knowingly non-compliant in one country. As data, a change is a configuration publication that takes effect in seconds — and is therefore a path by which behaviour changes with none of the review that code gets. The second risk is the real one, and it is manage; the first is not, because the platform does not control when a regulator acts. There is a second question underneath: when the rule set cannot be read, does the platform send or refuse? Refusing blocks legitimate traffic; sending risks a fine and a suspended sender identity, which blocks all of the tenant's traffic for longer.

**Decision.** Jurisdictional rules are versioned control-plane data, published through a reviewed path, effective in the data plane within 30 seconds, and rolled back by moving a version pointer. Every compliance refusal records the rule-set version consulted. The data plane runs on a cached rule set with a bounded staleness ceiling, and past that ceiling compliance fails closed while everything else in the platform continues to degrade gracefully.

**How it is realised on the stack.** Rules live in Spanner as versioned rows and are cached in Memorystore with an explicit invalidation path and a staleness ceiling measured in seconds. The publication path is reviewed like code — a change request, a diff against the live version, a blast-radius preview naming the tenants and routes affected, and two-person approval — and publishes a new version plus a pointer move. The compliance evaluator stamps the version onto every refusal and every acceptance, so a dispute is answered with the rules as they were rather than as they are. Fail-closed is scoped deliberately: it applies to compliance only, while route quality, lease state and projections all fail open or degrade.

| Option | Verdict | Reasoning |
|---|---|---|
| Versioned data, reviewed publication, fail closed on staleness | Chosen | A regulator's change is live in seconds and auditable; a behaviour-changing path exists outside the code pipeline and needs equal review |
| Rules as code, shipped by deployment | Rejected | One review path and one audit trail, and a release cycle between a regulation changing and the platform obeying it |
| Versioned data that fails open on staleness | Rejected | Keeps traffic flowing and risks a fine and a suspended sender identity, which costs the tenant far more than the refused messages |
| Delegate jurisdictional compliance to each carrier | Right elsewhere | Right for a pure wholesale reseller; here the platform holds the tenant relationship and the regulator will ask it, not the carrier |

**What it buys**

- A regulatory change is live in 30 seconds without a release, which is the only tempo that matches the regulator's
- Every refusal is defensible years later, because the rule version consulted is on the record
- Rollback is a pointer move rather than a revert and redeploy
- Fail-closed is scoped to one evaluation, so the rest of the platform keeps its graceful degradation

**What it costs**

- A behaviour-changing path exists outside the code pipeline, and it needs the same review or the gate has only been moved
- A rule-set outage past the staleness ceiling stops sending for affected classes, which is a self-inflicted availability event
- Blast-radius preview is extra machinery that only earns its keep on the day somebody publishes a wrong rule

**Choose differently when.** If the rule set stabilised — a platform in three jurisdictions with settled regulation — code would be the better home, because one review path is simpler than two. It would also change if a trustworthy third-party compliance service emerged with a liability position the platform could rely on.

**Why it holds up over time.** Configuration that changes behaviour needs the review that code gets; the fact that it is data does not make it safe. That is a lasting principle, and so is the asymmetry underneath the fail-closed choice: refusing some traffic is recoverable, while a suspended sender identity is not recoverable on the platform's own timetable.

> **Lesson.** If data can change behaviour, it is code with a faster deployment pipeline. Give it the same review, or admit that the gate was removed rather than relocated.

#### ADR-14 · Receipts and consent outlive the bodies they describe, and store-no-body is a supported mode

**Status:** Accepted  ·  **Shown on views:** 10, 11, 21

*Message content, the record of what was done with it, and the proof of permission all have retention requirements. Do they share a policy?*

**Context.** One retention policy is simpler and wrong in both directions. Content is a privacy liability: tenants want it gone quickly, several want it never stored, and some jurisdictions require deletion. The record of what the platform did is an operational and commercial asset: a carrier invoice arrives weeks later, a dispute months later, and without the attempt and receipt history there is nothing to argue with. Consent and compliance evidence is a regulatory obligation on a different clock again, measured in years and not subject to the tenant's preferences at all. Collapsing these into one policy means either keeping content far longer than anyone wants, or destroying the evidence before it is needed.

**Decision.** Three clocks. Message bodies and media default to 30 days, are tenant-configurable from zero to 90, and a store-no-body mode is supported in which the platform retains metadata, attempts and receipts but never writes content. Attempts are 90 days hot and 13 months archived; receipts are 13 months regardless of body retention. Consent records, compliance verdicts and the audit log are seven years, independent of any tenant retention setting.

**How it is realised on the stack.** Bodies and media sit in Cloud Storage under per-tenant CMEK keys with lifecycle rules per tenant setting, physically separate from the Spanner ledger, so a purge touches no delivery record. Store-no-body is an architectural mode rather than a flag: the submit path passes content straight to the adapter and the retaining write never happens, so the promise is kept by the path not existing. Receipts land in their own partitioned store with their own lifecycle. Consent and audit go to write-once storage with Bucket Lock that the platform's own operators cannot shorten, which is what makes a seven-year claim credible.

| Option | Verdict | Reasoning |
|---|---|---|
| Three clocks: content shortest, evidence longest, store-no-body supported | Chosen | Each obligation gets the horizon it actually has; three lifecycle policies and three stores to reason about |
| One retention policy across all message data | Rejected | Simple, and it either keeps content too long or destroys billing and regulatory evidence before it is needed |
| Tenant-configurable retention for everything including evidence | Rejected | Maximally flexible, and it lets a tenant configure away the records a regulator will ask the platform for |
| Never store content at all | Right elsewhere | Right for a privacy-first product; here it removes template rendering, media handling and most debugging, so it is offered as a mode rather than imposed |

**What it buys**

- A body purge, including a complete store-no-body posture, leaves the delivery record and the receipts intact
- A carrier invoice dispute arriving ten months later has evidence behind it
- Consent and audit are credible for seven years because the storage itself prevents shortening
- Store-no-body is a strong privacy claim made structurally rather than by configuration

**What it costs**

- Three lifecycle policies across three stores, each a place to get retention wrong
- Receipts referencing content that no longer exists are normal, and tooling must handle that gracefully
- Store-no-body tenants lose template debugging and content-based support, which is a real product limitation

**Choose differently when.** A jurisdiction requiring deletion of all associated records, not just content, would force the evidence clock down and change the dispute position accordingly. If carriers moved to real-time settlement, the thirteen-month receipt horizon could shorten substantially.

**Why it holds up over time.** Content, the record of acting on content, and the proof of permission to act are three obligations to three different parties with three different horizons, and that will not converge. A system giving them one policy is wrong about at least two, and discovers which only when asked.

> **Lesson.** Retention is not one decision. Separate the thing, the record of what was done with it, and the proof it was allowed — then let each follow the party that will ask for it.

### Cost and operability

*What a message actually costs, and how a route earns the right to carry one.*

#### ADR-15 · One rated usage event per attempt, not per message

**Status:** Accepted  ·  **Shown on views:** 10, 14, 19

*A message with a fallback chain may be attempted on two channels at two prices, and a long body is several segments. What is the unit of charge?*

**Context.** Rating per message is what tenants expect and what the pricing page implies, and it cannot represent what actually happened. A fallback from a rich channel to SMS is two carrier calls with two prices. A re-route after a failure is another. A 400-character body is three segments. Rating per message forces the platform to choose one of them as canonical and lose the rest, which makes margin unknowable per route and makes a carrier invoice impossible to reconcile — the invoice is per carrier call, because that is what the carrier charged for.

**Decision.** A rated usage event is emitted per attempt, carrying tenant, sender identity, route, channel, traffic class, segment count and amount. The message is a grouping for the tenant's benefit, not the unit of charge. Downstream metering and invoicing decide how to present it; this platform's responsibility ends at an accurate, attributable event per carrier call.

**How it is realised on the stack.** The correlate-and-apply workers emit one event per attempt to a Pub/Sub topic consumed by the metering platform, with the ledger as the replayable source if the stream is lost. Events are attributed to tenant, sender identity, route, channel and class, which is what makes margin knowable per route rather than only in aggregate, and what lets the platform publish price and observed quality together. Because the ledger holds every attempt, the usage stream is reconstructible, so metering gaps are recoverable rather than permanent revenue loss.

| Option | Verdict | Reasoning |
|---|---|---|
| One rated event per attempt | Chosen | Reconciles to the carrier invoice and makes per-route margin knowable; the tenant's bill needs explaining when a fallback charged twice |
| One rated event per message, summing attempts | Rejected | Matches tenant intuition, and loses the route attribution that reconciliation and margin analysis both need |
| One event per segment | Rejected | Finest grain and closest to some carrier models; multiplies event volume for information the attempt already carries |
| Rate inside this platform rather than emitting usage | Right elsewhere | Right where there is no separate metering platform; here it duplicates a system that exists and owns pricing |

**What it buys**

- The usage stream reconciles line by line against a carrier invoice, which is what makes a dispute winnable
- Margin is knowable per route, which is what makes least-cost routing a decision rather than a hope
- A fallback chain's true cost is visible to the tenant and to the platform
- The stream is reconstructible from the ledger, so a metering outage is recoverable

**What it costs**

- Event volume follows attempts, not messages, so retries and fallbacks inflate it
- A tenant seeing two charges for one message needs an explanation that the API and the console must both give
- Attempt-level attribution has to be correct at submit time, because it cannot be reconstructed from content later

**Choose differently when.** If the platform ever sold a flat per-message price absorbing all attempts, the internal event grain could stay and the external presentation change — which is in fact the intended seam. Only a move to per-segment carrier billing across the whole estate would argue for a finer grain.

**Why it holds up over time.** The unit of charge should match the unit of cost, and the unit of cost is whatever the supplier invoices. That alignment keeps reconciliation possible and margin visible, and it survives every change of pricing model because presentation can always aggregate upward and never disaggregate downward.

> **Lesson.** Measure at the grain of the cost, present at the grain of the customer. Those are different decisions, and conflating them destroys information that cannot be recovered.

#### ADR-16 · Route eligibility is earned from measured delivery, and absence of receipts is a signal

**Status:** Accepted  ·  **Shown on views:** 19, 20

*A carrier's contract promises a delivery rate. Should the platform route on that promise, or on what it can measure?*

**Context.** Contracted quality is a sales document. The failure mode that matters most in this business is a route that accepts every submission and delivers nothing — no errors, no rejections, a perfect acceptance rate and a collapsed delivery rate — and it is invisible to any monitoring that watches requests rather than outcomes. A related failure is quieter still: a route that simply stops reporting receipts, which under an optimistic status model looks like improved delivery and under this platform's model looks like a rising unknown share. Both are commercial problems disguised as technical silence, and neither is discoverable without routing on measurement.

**Decision.** A route's eligibility and weight come from measured outcomes — delivery rate, receipt latency, unknown share and acceptance-failure rate — recomputed continuously from the receipt stream. A route with no measured outcome cannot remain eligible: the absence of receipts is treated as a signal, not as missing data. Breakers trip automatically for interactive traffic and advise for bulk, because re-routing bulk spends the tenant's money. Retirement is explicit: a route is requalified or removed, never left in the table at zero weight.

**How it is realised on the stack.** Dataflow aggregates the receipt stream into per-route windows in BigQuery and publishes current figures to the Memorystore quality store the selector reads. A delivery-rate collapse detector compares a route's current window against its own baseline rather than against a global threshold, so a route that has always delivered 70% is not treated like one that fell from 98%. Price and observed quality are always reported together, because least-cost routing that halves delivery rate is not a saving. The lifecycle is a loop with an explicit end state, so a dead route leaves the table rather than lingering at weight zero where it will be forgotten.

| Option | Verdict | Reasoning |
|---|---|---|
| Eligibility from measured outcomes, no-receipts as a signal | Chosen | Catches silent collapse and silent non-reporting; quality signals lag price changes, so a cheap bad route can win briefly |
| Route on contracted quality with manual review | Rejected | Trusts a sales document and makes silent collapse a customer-reported incident |
| Route on measured quality but treat missing receipts as neutral | Rejected | Rewards carriers for not reporting, which is the easiest quality metric in the world to game |
| Let tenants choose routes themselves | Right elsewhere | Right for wholesale customers with their own carrier relationships; offered as a pinned-route policy, not as the default |

**What it buys**

- Silent delivery collapse becomes an automatic failover for interactive traffic rather than a support ticket
- A carrier that stops reporting loses traffic, which turns a technical silence into a commercial conversation
- Baselining per route avoids punishing a route that was always mediocre and rewarding one that just broke
- Retirement is explicit, so the route table reflects reality rather than history

**What it costs**

- Quality signals lag price changes, so a least-cost policy can pick a cheap bad route faster than the loop discovers it
- A new route has no measurements, so onboarding needs a probationary period rather than immediate eligibility
- Automatic failover spends more money without asking, which has to be bounded and reported

**Choose differently when.** If carriers provided trustworthy, audited, real-time quality telemetry, the platform could route on their figures and keep its own measurement as verification. Conversely, if the estate shrank to a handful of directly contracted routes, manual review with measurement as evidence would be sufficient and cheaper.

**Why it holds up over time.** Supplier quality should be measured by the buyer, because the supplier's incentive to report is weakest exactly when the report matters most. Treating silence as information, rather than as absence of information, is the part that most often gets lost, and it is the part that catches the worst failures.

> **Lesson.** Never let a metric improve when a supplier stops reporting. If silence can look like success, the measurement is pointed at the wrong thing.

## Every package used, in one table

Twelve terms used precisely throughout the package, several of which mean something narrower here than in ordinary usage.

| Package | What it is | What it does here | Considered instead |
|---|---|---|---|
| Accepted | The state of a message whose submission has been validated, screened and durably appended to the ledger, and acknowledged to the caller. | The platform's promise of ownership, and the only thing the submit path ever promises. | Often read by tenants as "sent", which it deliberately is not — hence the separate vocabulary in the API. |
| Attempt | One carrier call for one message, with its own route, segment breakdown, provider reference, outcome and price. | The unit of retry, of routing, of rating and of the fold that produces delivery state. | Sometimes called a delivery or a send, both of which imply an outcome the attempt does not carry. |
| Receipt | An asynchronous statement from a carrier about an attempt, arriving seconds to hours later, possibly duplicated, out of order, or for an unknown reference. | Evidence. It is appended and retained for thirteen months, longer than the body it describes. | A delivery report or DLR in carrier protocols, which overstates how definitive it is. |
| Unknown | The terminal state of a message whose time-to-live elapsed with no conclusive receipt. | A first-class answer in the API, the SLOs and the route-quality signal — never collapsed into delivered or failed. | Usually reported elsewhere as delivered, which is the lie this platform is designed not to tell. |
| Ambiguous submit | An attempt that timed out with no response on a route offering no idempotency key, so neither delivery nor non-delivery can be inferred. | A recorded fact resolved by per-route policy, with the policy applied written onto the message. | Commonly treated as a plain timeout and retried, which is how duplicate login codes are made. |
| Route | A tuple of channel, destination jurisdiction, provider and sender-identity type, carrying price per segment, a contracted throughput window, supported features, observed quality and health. | The unit of pacing, failover, pricing, policy and commercial relationship. | Loosely called a carrier or a connection, which conflates the provider with the specific lane bought from them. |
| Lease | A block of permits checked out from a route's contracted throughput window and spent locally without further coordination. | How a contractual window is held across an elastic fleet with no coordination hop per message. | A rate limit, which understates the penalty: breaching a window suspends a sender identity. |
| Sender identity | The thing a recipient sees as the origin: a long number, a short code, an alphanumeric sender ID, a toll-free number or a channel business identity, with the jurisdictions it is registered in. | The asset a throughput breach or a compliance failure actually damages, and the key by which inbound traffic finds its tenant. | Often called the "from" number, which hides that it is a registered, suspendable entity rather than a field. |
| Traffic class | One of interactive, conversational or bulk, declared at submission. | Decides the lane, the route preference, the SLO and the position in the shedding order. | A priority, which implies a single ordering rather than genuinely different products. |
| Client reference | A tenant-supplied identifier, unique per tenant, carried on the submission. | The idempotency key that makes a resend return the original message, and the only identifier a tenant's support desk has. | An external id or correlation id; here it has enforced uniqueness and a dedup window, which those usually do not. |
| Regulated keyword | An inbound message such as STOP, UNSUBSCRIBE or HELP, or a locale equivalent, whose handling is mandated rather than optional. | Intercepted by the platform before the tenant is told, with the consent change written first and a mandated reply sent. | Treated elsewhere as tenant application logic, which is how a STOP on a shared number ends up unenforced. |
| Projection | A rebuildable read model derived by folding a message's attempts and receipts, served for status queries. | The thing tenants read. It may lag, must be rebuildable in hours, and is never the authority. | A status table, which looks the same and is authoritative, which is the difference this package is about. |