# Webhook Delivery Service

**Solution Architecture v1.0 · Amazon Web Services · Integration Platform Architecture · 2026-09 · 20 views · 16 architecture decision records**

The mechanism behind every `payment_intent.succeeded`, `pull_request.opened` and `order/create` a SaaS product ever sent: the service that turns a committed domain event into a signed HTTPS request against 40,000 endpoints nobody here controls, keeps trying for 72 hours when one of them is having a bad day, and lets the developer who owns it find out why theirs is not receiving anything — without opening a support ticket. It handles 25,000 accepted events per second and 45,000 delivery attempts per second across two regions, built from SQS FIFO for per-endpoint isolation and redrive, Fargate delivery workers in egress-only subnets behind a published NAT range, DynamoDB for subscriptions and the attempt log, S3 for payloads, and KMS for per-endpoint signing material.

The design rests on one rule: **capture and delivery are separate planes joined by a durable log, and the unit of isolation on the delivery side is the endpoint.** Not the tenant, not the event type, and not a shared worker pool.

The decisions that carry the design:

- **The product's write path ends at durability.** Intake persists and acknowledges without consulting subscription state, so adding 500 endpoints cannot make checkout slower. Everything after it happens on the platform's own time (ADR-01).
- **The endpoint is the blast radius.** Two endpoints belonging to the same customer have independent queues, circuits, backlogs and health. The failure being designed against is a consumer that accepts the connection and responds in fifteen seconds without ever erroring (ADR-02).
- **At-least-once, said out loud.** A stable idempotency key per delivery, constant across every retry and every replay, and a published contract that puts deduplication where it can actually be done (ADR-04).
- **Ordering is declared, not assumed.** Strict per-endpoint ordering turns one poison payload into a 72-hour outage for that customer. It is offered with the consequence attached, and the default is none (ADR-05).
- **The signature is a contract, not a description.** A published canonical string frozen by a CI test, two active secrets with a seven-day overlap, and worked verification examples — because a signature nobody can verify is decoration (ADR-06, ADR-07).
- **The retry schedule belongs to the platform.** Twelve attempts over 72 hours with full jitter, `Retry-After` honoured, and resumption ramped from 10% to 100% over five minutes so the platform is never the cause of the consumer's second outage (ADR-08).
- **Give up loudly.** Notification at the *first* dead letter, not the hundredth, on a channel that does not depend on the customer's endpoint working. An integration that stops silently is the worst thing this platform can produce (ADR-09).
- **The network contains the worker, not the guard code.** A delivery worker is a request-forgery engine aimed at a URL a stranger chose. Address validation runs before every connect, and egress-only subnets with no inward route are what holds when it has a bug (ADR-12).
- **Two systems of record.** What we were asked to send, and what we actually did. Neither is derivable from the other, and the queues are rebuilt from the second (ADR-15).
- **An acknowledged unsubscribe binds.** Strongly consistent subscription reads at fan-out, and a URL change — which redirects a tenant's data — needs a stronger proof than changing which event types are sent (ADR-16).

The architecture one-pager (including why the design should still hold up in ten years) and the full decision record appear on the landing page of the diagram set, directly below the index of views. The same content is published as [docs/architecture-one-pager.md](docs/architecture-one-pager.md) (~12 min) and [docs/decision-record.md](docs/decision-record.md) (~58 min).

---

## What is here

| Path | Contents |
|---|---|
| `diagrams/index.html` | The landing page: 20 views in seven acts with every format linked, then the **architecture one-pager** and the **decision record** |
| `diagrams/*.html` | One self-contained page per view: the inlined diagram plus the reasoning cards, with copy / PNG / PDF export |
| `diagrams/svg/*.svg` | The same views as SVG with the diagram XML embedded; they re-open fully editable in diagrams.net |
| `diagrams/drawio/*.drawio` | draw.io source |
| `docs/architecture-one-pager.md` | The one-pager as markdown |
| `docs/decision-record.md` | The 16 decision records, the capability-to-technology table and the package glossary as markdown |
| `specs/part-a..d.json` | Diagram specifications, the source of truth for every view |
| `specs/manifest-a..b.json` | Acts, page titles, subtitles and reasoning cards |
| `specs/adr-onepager.json`, `specs/adr-records-a..b.json` | The one-pager, the decision records, the capability-to-technology table and the glossary |
| `scripts/build.sh` | Rebuilds every deliverable from the specs (Node only) |
| `ask.md` | The requirement |

## The twenty views

| # | View | What it answers |
|---|---|---|
| 01 | System Context | Who the platform talks to, and the four things it refuses to own |
| 02 | High-Level Architecture | Seven stages from a committed event to an accounted-for delivery |
| 03 | Actors and Journeys | Six actors, and what each of them gets to do |
| 04 | Journey — First Webhook | An integration developer's afternoon, and where it always stalls |
| 05 | Journey — Endpoint Outage | Their receiver dies at 02:00; everything the platform does next |
| 06 | Layered Architecture | Seven layers, and what each of the middle ones is forbidden to know |
| 07 | Container View | Capture, fan-out, delivery and control, and which subnet each lives in |
| 08 | Integration Surface | Every interface the platform commits to, inbound and outbound |
| 09 | Storage Zones | Four zones divided by what can be rebuilt and what cannot |
| 10 | Data Model | Eight entities, and the one field the consumer contract depends on |
| 11 | Critical Flow — Event to First Attempt | Seventeen messages, and the one that ends the product's involvement |
| 12 | Retry, Circuit Breaking and Dead Letter | Four outcome classes, one published schedule, two ways back in |
| 13 | Signing, Rotation and Verification | Key generation to the consumer's comparison, including the overlap |
| 14 | Replay and Recovery | Five lanes, and why bulk replay needs a confirmation |
| 15 | Deployment Architecture | Two regions, four subnet tiers, an egress range that is an interface |
| 16 | Release Pipeline | Three gates that exist because of what this platform is |
| 17 | Observability | Five signals across five stages, and the row the customer sees |
| 18 | Endpoint Health Lifecycle | Six states, per endpoint, and the transition a human must make |
| 19 | Trust Zones and the Egress Path | A worker fleet aimed at strangers' URLs, and the five zones around it |
| 20 | Identity and Access | Why changing a URL needs a stronger proof than changing an event type |

## Rebuilding

```bash
bash scripts/build.sh
```

Node 20+ and nothing else — no draw.io Desktop, no browser, no network. The build regenerates
every `.drawio`, both SVG forms, all 21 HTML pages and both markdown documents from the specs,
and fails on a geometry error, a routing error (an arrow crossing a box, a label sitting on
one), a broken relative link, or a decision record that references a view that does not exist.

Current state: **20 views, 291 icons embedded, 0 geometry errors, 0 routing errors, 0 warnings,
all relative links resolve.**