Document 16 min read

Architecture One-Pager

Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-10 · 22 views · 16 architecture decision records

Programmable Messaging API · Solution Architecture v1.0 · Google Cloud · Integration Platform Architecture · 2026-10 · 22 views · 16 architecture decision records

The accepted message is the unit of durability and the carrier attempt is disposable: delivery state is a monotonic projection over attempts and late-arriving receipts, never a field the submit path writes.

Everybody has used this system today without knowing its name. The six-digit code that arrives when you log into your bank, the "your driver is 2 minutes away" text, the prescription reminder, the airline's gate-change message. None were sent by a human, and almost none by the company whose name is on them. They went through a programmable messaging API: one HTTP endpoint a developer calls with a destination, a body and a sender identity, and a platform behind it that turns that call into a message on a handset anywhere in the world, through carriers it does not own, under regulations it does not write. The product surface is one POST. The architecture is hard for exactly one reason, and it is not scale: the platform cannot see whether the message arrived. It hands the message to a carrier or an aggregator, and what comes back is a receipt that may arrive in two seconds, in four hours, or never; that may say "delivered" when the handset was off; that may arrive twice, out of order, or for a message the platform has no record of. Everything difficult here follows from that: what the acknowledgement at submit is allowed to promise, what happens when a submit times out with no answer, whether a status field or a fold over facts decides the truth, how much carrier capacity may be spent before a sender identity is suspended, and who is accountable when somebody texts STOP to a number four tenants share.

The design rests on one boundary. An accepted message is owned: it is validated, screened against consent and jurisdictional rules, and appended to a strongly consistent ledger before the API acknowledges anything, and the acknowledgement promises ownership rather than delivery. Everything after that point is disposable. A carrier attempt is a separate append-only record with its own route, segments, price and outcome, so a 503 is another attempt, a 429 is a park with a release time, a dead session is a resume, and a worker's death costs nothing. Delivery state is never written by the submit path and never stored as a mutable column: it is a fold over the attempt series and the receipts correlated to it, which is what makes a receipt arriving four hours late, twice, out of order, or after the body has been deleted a fact to append rather than a race to lose. Because the fold over "one attempt accepted, no receipt, deadline passed" is unknown, the platform's most unwelcome product requirement becomes structurally unavoidable rather than a matter of downstream discipline. Around that spine sit four things the platform does not control and must therefore model explicitly: carrier throughput windows, enforced by per-route leases because breaching one suspends a sender identity rather than merely rate-limiting it; traffic classes, because an OTP and a four-million-message campaign are different products sharing only the ledger; channel adapters, so that adding RCS is an adapter and a route family rather than a change to the state model; and a stateful session tier for long-lived carrier binds, which fit neither the elastic fleet above them nor the carriers below.

What it is, and what it is not

  • An owner of accepted messages and a recorder of what it tried — not A reporter of whether a handset received anything
  • A fold over attempts and receipts — not A status column somebody updates
  • A platform where unknown is an answer — not A platform that rounds unknown to delivered
  • A courier inside other people's throughput windows — not A system that retries until something works
  • The owner of regulated keywords and consent evidence — not A pass-through that trusts the tenant's opt-out list
  • Three traffic classes that share only the ledger — not One queue with a priority field
  • A thin adapter contract with declared per-channel capability — not A lowest-common-denominator message that silently degrades
  • A keeper of receipts longer than the bodies they describe — not A store with one retention policy

The decisions that are the architecture

  1. The accepted message is owned; the carrier attempt is disposable (ADR-01) — Two durable writes per attempt buy a system where every carrier failure is the same mechanism: appending a fact. The alternative pays by re-running what already succeeded.
  2. Delivery state is a projection, not a stored field (ADR-02) — Out-of-order receipts are the normal behaviour of carrier networks, not an edge case. A column loses that race silently; a fold cannot.
  3. Accept before route (ADR-03) — Submission never waits on a route, a session or a carrier, so the two fail separately and the submit SLO is achievable without any of them being healthy.
  4. Unknown is a first-class terminal state (ADR-04) — A stored status always has a tempting default for "we heard nothing". A projection has none, so the honesty is enforced by structure rather than by discipline.
  5. The ambiguous submit is resolved by recorded per-route policy (ADR-05) — No idempotency key means choosing between a duplicate code on a handset and a silent non-delivery. The choice differs by route and class, and is written onto the message.
  6. Uncorrelatable receipts are quarantined, never discarded (ADR-06) — A receipt matching nothing is usually a platform bug and occasionally the evidence in a billing dispute. Dropping it destroys both.
  7. Per-route throughput leases, checked out in blocks (ADR-07) — A carrier window is contractual: breaching it suspends a sender identity. Correctness here is worth a bounded overshoot and a lease that can be stranded.
  8. Three classes with a declared shedding order (ADR-08) — Interactive, conversational and bulk get separate lanes and separate SLOs, and load is shed by refusing in a stated order rather than by making everything slower.
  9. Routing decides at attempt time (ADR-09) — A route chosen at submit may be dead when the attempt runs. The price quoted at submit becomes an estimate, and that is said out loud in the API.
  10. A thin adapter contract with declared capability (ADR-10) — Channels differ in ways tenants need to see. Hiding the differences makes the platform responsible for a translation nobody asked for.
  11. A dedicated stateful session tier (ADR-11) — Long-lived binds with sequence windows and allowlisted egress do not belong in an elastic fleet where every scale-in is a teardown and an in-flight ambiguity.
  12. The platform owns regulated keywords (ADR-12) — STOP on a shared number cannot be a tenant preference. The platform actions it, then tells the tenant it has already been actioned.
  13. Jurisdictional rules are versioned data and fail closed (ADR-13) — A rule change must not need a deployment, and a message the platform is unsure it may send is not sent. Everything else degrades.
  14. Receipts outlive the bodies they describe (ADR-14) — Content retention is a privacy decision; receipt retention is an evidence decision. Collapsing them means disputes arrive after the proof has gone.
  15. One rated usage event per attempt (ADR-15) — A fallback hop, a re-route and a multi-segment body each cost money independently, so the message is the wrong unit to rate.
  16. Route eligibility is earned from measured delivery (ADR-16) — Contracted quality is a sales document. A route with no measured outcome cannot stay eligible, because absence of receipts is itself a signal.

Why this should still be right in ten years

SMPP, the aggregator of the month, Spanner, Bigtable, the channel that replaces RCS and probably the regulator's current rulebook will all be replaced inside this platform's life. These five properties should survive the replacement, and each is a statement about where something lives rather than about what it is built from.

  • Truth is a fold over facts, not a field. Any system whose outcome is reported asynchronously by a party it does not control faces the same race, and the race gets worse as the number of reporting parties grows. A projection over append-only facts is indifferent to arrival order, which is the one property that does not degrade with scale, with a new channel, or with a carrier whose receipts are worse than the last one's.
  • The unit of durability is the promise, not the attempt. What the platform promised a caller and what it tried on their behalf are different lifetimes with different owners. Keeping them separate is why a retry, a park, a re-route and a worker death are one mechanism rather than four special cases, and it will still be why when the protocol underneath has changed twice.
  • What the platform cannot know is part of the product. Naming the unknown rather than rounding it away is a design stance, not a feature. It survives because the alternative compounds: once a status field has been allowed a comfortable default, every downstream report, SLO and routing signal inherits the lie, and no later component can recover the distinction.
  • Somebody else's capacity is a design input. The platform will always be spending throughput it rents under terms it did not set. Modelling that as a lease with a release time, rather than as an error to retry, keeps pacing a normal state and keeps the platform's relationship with its suppliers intact — whoever the suppliers are by then.
  • Evidence is retained on its own clock. The retention of content, of the record of what was done with it, and of the proof of permission are three different obligations with three different time horizons and three different regulators. A system that gives them one policy will be wrong about at least two, and the mistake is only discovered when somebody asks.

Non-functional targets

Every target below is a stated assumption, sized for the operating context in the evidence note and chosen so a reviewer can argue with a number rather than with a sentiment. Each names the view where the mechanism that delivers it is drawn.

Quality Target How it is met View
Submit availability ≥ 99.99% monthly Stateless submit across zones, one durable ledger write, no dependency on any route, session or carrier being healthy 13
Receipt and inbound ingest availability ≥ 99.99% monthly Verify, make durable, acknowledge — correlation happens afterwards, so ingest never waits on the ledger being queryable 14
Consent evaluation availability ≥ 99.99% monthly Strongly consistent store read on every submit, cached with a bounded staleness ceiling, failing closed past it 21
Submit acknowledgement latency p95 ≤ 150 ms, p99 ≤ 400 ms One synchronous ledger commit; validation, sender and consent checks are local reads or cached reads 13
Interactive time to carrier acceptance p50 ≤ 700 ms, p95 ≤ 2 s Dedicated interactive lane, per-route lease taken locally from a checked-out block, no cross-region hop 15
Receipt visibility p95 ≤ 2 s from carrier receipt Stream-driven correlation and fold, with the projection written ahead of the callback dispatch 14
Throughput 8,000 msg/s steady, 40,000/s for 10 min Independent scaling of submit, scheduling, attempt and ingest; ledger partitioned by tenant and time 17
Durability of accepted messages RPO 0, zero lost Synchronously replicated ledger; the acknowledgement is issued only after commit 11
Projection recovery RTO 2 h to rebuild State is a fold, so the projection is disposable; status queries degrade rather than fail during a rebuild 10
Duplicate handset deliveries ≤ 1 per 10,000,000 accepted Client-reference idempotency at submit, a single attempt key per carrier call, and a default of not re-attempting an ambiguous submit 16
Consent correctness zero sends after withdrawal, outside 30 s Inbound keyword interception writes consent before the tenant is notified; propagation bound is stated, not implied 06
Fairness interactive queue age p99 ≤ 3 s Class lanes first, then per-tenant share capped at 40% of a shared route while others have work queued 19
Cost platform ≤ 8% of carrier segment cost Ledger write path held to ≤ 35% of platform cost; dormant tenants and idle routes near zero 19

Scope

In scope

  • Message submission: validation, client-reference idempotency, segment counting, time-to-live, traffic class, batch and dry-run
  • Destination normalisation, sender-identity registry and jurisdictional sender rules as versioned data
  • Routing across carrier and aggregator routes with cost, quality and pinned policies, per-route breakers and automatic interactive failover
  • The append-only attempt series, retryability classification, bounded retry inside the time-to-live, and a recorded per-route ambiguous-submit policy
  • Receipt ingestion: verification, dedup, correlation, quarantine of the uncorrelatable, and the monotonic state fold including unknown
  • Status by pull and by signed callback with retry and a dead-letter path, reconstructible entirely from pull
  • Inbound messages, per-route dedup, tenant routing by sender identity, and regulated keyword interception with immediate consent enforcement
  • Consent records and jurisdictional compliance on the submit path, failing closed, with append-only history and per-destination evidence
  • Per-tenant entitlement, traffic-class scheduling, cross-tenant fairness, credit checks and suspension
  • Channel adapters for SMS, MMS and at least one rich channel, behind one internal contract, plus the stateful carrier session tier
  • One rated usage event per attempt, and the audit log that records every configuration and access event

Explicitly out of scope

  • Metering, rating tables and invoicing — this platform's output is the rated usage event; see usage-metering-and-billing-pipeline
  • Generic outbound webhook fan-out to tenant systems; see webhook-delivery-service
  • Application notification orchestration and end-user channel preference logic; see event-driven-notification-platform
  • Standalone request deduplication as a shared service; see idempotency-and-dedup-service
  • Voice and video channels, and the number-provisioning marketplace — phone numbers are an inventory this platform reads
  • Campaign authoring, audience segmentation and marketing analytics
  • The tenant's own identity provider, and end-user account recovery inside the tenant's product

What a four-week prototype should prove

The prototype's job is to falsify the three claims the design rests on: that a fold over attempts and receipts is cheap enough to answer status queries at rate, that per-route lease enforcement holds a contractual window across an elastic fleet without a cross-region hop, and that the ambiguous-submit default produces fewer bad outcomes than re-attempting. Everything else can be adjusted; if any of those three is wrong, the boundary in ADR-01 has to move.

  1. Two routes in two jurisdictions through two different providers, one SMPP and one HTTP, because a single provider hides every difference the adapter contract exists for
  2. A receipt generator that is deliberately hostile: late by hours, duplicated, out of order, occasionally for an unknown reference, and silent for a configurable share of traffic
  3. The full fold implemented against the ledger with no status column anywhere, and status queries measured at 10x the expected rate while the projection is rebuilt
  4. A contractual window of 300 messages per second enforced across a fleet that scales from 2 to 40 workers during the test, with overshoot measured rather than assumed
  5. An ambiguous-submit harness that drops responses after the carrier has accepted, so the duplicate rate of the re-attempt policy can be compared with the unknown rate of the default
  6. One interactive tenant and one bulk tenant on the same route, with the bulk tenant submitting four million messages while interactive queue age is measured
  7. An inbound STOP delivered twice, out of order, on a sender identity shared by two tenants, with the consent write and both tenants' subsequent submits observed
  8. Deliberately not in the prototype: the console, more than two channels, billing integration, and any form of campaign tooling
  • Twenty-five thousand receipts a second are replayed against the fold while the Bigtable projection is rebuilt from the ledger, with status queries served throughout at ten times the expected rate: every message's final state matches the state computed from the same facts in arrival order, and no query fails
  • A 'delivered' receipt generated at 10:00 arrives after an 'undelivered' generated at 09:58, four hours after the message's deadline and after its body has been purged: the message is already unknown, the late receipt is retained as evidence, and no transition out of the terminal state occurs
  • A route with a contracted window of 300 messages per second is driven while the attempt fleet scales from 2 to 40 workers and back twice: measured peak throughput never exceeds the window by more than the configured block size, and stranded capacity from killed workers is visible as a throughput dip rather than as a breach
  • The carrier harness accepts a submission and then drops the response: the ambiguous outcome is appended, the per-route policy applies and is recorded on the message, and the resulting duplicate rate under a re-attempt policy is measured against the unknown rate under the default so the ADR-05 trade is argued with numbers
  • A bulk tenant submits four million messages on the same route a second tenant is using for login codes: interactive queue age stays within its budget throughout, and under induced overload bulk is refused before conversational and conversational before interactive
  • An SMPP bind is killed uncontrolled with forty messages in flight, then deliberately drained with forty in flight: the uncontrolled case produces exactly forty ambiguous attempts resolved by policy, and the drained case produces none
  • A STOP arrives twice, out of order, on a sender identity shared by two tenants: one consent event is written, one mandated confirmation is sent, both tenants' subsequent submits are refused with the consent version cited, and messages already accepted when it landed are counted and reported rather than silently delivered
  • A receipt arrives carrying a provider reference the platform never issued: it is quarantined with its payload, the quarantine rate-of-change alert fires, and after a simulated correlation fix the window is replayed and the affected folds recomputed

Open risks, carried rather than hidden

Risk If it lands Response
The ledger write path is the platform's cost and scale ceiling Two synchronously replicated writes per attempt at 8,000 messages a second with retries and fallbacks is the hottest path in the system; if it exceeds its stated share of per-message cost the design is uneconomic rather than merely expensive Price it explicitly as a reported metric per message rather than per month, hold it to the 35% share in the NFRs, and treat batching of intent writes as the first optimisation to evaluate — never by weakening the intent-before-call ordering
The ambiguous-submit default is a judgement a real customer may reject Not re-attempting protects a recipient from a duplicate code and accepts a silent non-delivery, which for a bank's login traffic may be exactly the wrong trade Keep it a per-route and per-class policy with the choice recorded on each message, publish the measured duplicate and unknown rates per route, and let a tenant's traffic class select the other default where they can defend it
Unknown-state share will be read as a platform defect Routes that never report receipts inflate the unknown rate for reasons outside the platform, and the honest number is a worse-looking number than a competitor's rounded one Report unknown share per route alongside the carrier's own reporting behaviour, make it a routing input so bad reporters lose traffic, and state the distinction in the product rather than in a support article
Carrier egress allowlists make regional expansion a correspondence exercise Standby binds in a second region need their source addresses approved by every carrier before failover works, on the carrier's timetable, not the platform's Treat the allowlist as a tracked dependency per route with its own state, refuse to count a region as a failover target until binds there have carried real traffic, and rehearse on a schedule rather than on an incident
Consent on shared sender identities is a liability concentrator One STOP on a number four tenants share must silence all four, and the platform — not the tenant — will be asked why it did not Keep regulated keywords platform-owned and the consent history append-only and queryable per destination, publish the 30 s propagation bound as a commitment, and make in-flight messages after a withdrawal a reported number rather than an unexamined gap
The quarantine of uncorrelatable receipts becomes a queue nobody works A rising quarantine rate is the earliest signal of a correlation defect and the easiest dashboard to stop reading, and the window in which a billing dispute can be answered closes quietly Give the quarantine an owner and an age distribution rather than a count, alert on the rate of change rather than the level, and require a named cause before any quarantined receipt ages out

The reasoning behind every component and technology choice is in the Architecture Decision Record: 16 records across 6 areas, each with the alternatives that lost and what the choice costs.