Document 14 min read

Architecture One-Pager

Solution Architecture v1.0 · Amazon Web Services · Security Platform Architecture · 2026-09 · 21 views · 20 architecture decision records

Certificate Lifecycle Service · Solution Architecture v1.0 · Amazon Web Services · Security Platform Architecture · 2026-09 · 21 views · 20 architecture decision records

The issuance log is not the truth. The endpoint is. A certificate is SERVING because something was observed presenting it, never because an issuance succeeded.

A mid-size B2B SaaS depends on two populations of certificate that behave nothing alike. Twenty-four thousand customer-owned domains are pointed at the platform by tenants who expect a padlock in browsers the platform has no relationship with; three thousand eight hundred internal workloads authenticate each other over mTLS under a certificate authority the platform runs itself. Both populations expire. Expiry is the most preventable failure in production software — a known date, in a machine-readable field, on a certificate the company usually issued to itself — and it still takes down household-name services every year. It does so for a reason that is architectural rather than operational: almost every certificate lifecycle system treats successful issuance as successful renewal. The platform issues, writes the new certificate to a secret store, marks the task complete, and moves on, while the process that actually terminates TLS is still holding the old certificate in memory, having never been told to reload. The issuance log says green. The endpoint says thirty days. Nothing errors, nothing alerts, and the gap between the two is where the outage lives.

A managed identity is the durable object: it carries an owner, a trust domain, a subject and a compromise history, and it outlives every certificate it holds. Entitlement to a certificate is proved differently per trust domain — a one-time CNAME delegation for a customer's public domain, an attested platform credential for a workload — and authorised by policy before any signature is requested. Issuance is the only plane that can cause a signature: an ACME client against two pre-validated public CA accounts on one side, AWS Private CA under CloudHSM custody on the other. Certificates are delivered through a small set of supported adapters, with the reload trigger inside the renewal transaction rather than downstream of it. Then the design turns: an independent verification plane, deployed per region outside the control plane's failure domain, opens a TLS connection to every known endpoint on a fifteen-minute cadence and writes what it actually saw. That write is the only one that moves a certificate into SERVING, and the renewal scheduler reads what is due from observed state rather than from issuance records — so a renewal that did not take effect stays due. Around it sit the consequences: a registry that is a projection rather than a ledger, discovery and CT monitoring as inputs to the same state machine rather than an audit feature, a trust plane gated strictly ahead of issuance, and a revocation path published independently of everything else.

What it is, and what it is not

  • A record of what is being served — not a record of what was issued, trusted to still be true.
  • A platform whose renewal completes when a process reloads — not one whose renewal completes when a CA responds.
  • One inventory across two trust domains — not one issuance mechanism across two trust domains — they share almost nothing.
  • A system that treats a failed renewal at T-30 as an incident — not a cron job whose failures become incidents at T-0.
  • A certificate authority operator — not a TLS terminator — it supplies and verifies the edge and the mesh, and owns neither.
  • A platform that never holds a private key — not a platform that holds private keys carefully.
  • Fail static under its own outage — not fail closed — nothing already serving is ever invalidated by this platform being down.
  • Sized to an external revocation deadline — not sized to its own convenience, then surprised by a CA's 24-hour clock.

The decisions that are the architecture

  1. The endpoint is the system of record (ADR-01) — Lifecycle state is derived from what endpoints are observed presenting. The issuance log records what was attempted; only an observation records what is true.
  2. ISSUED and SERVING are different states (ADR-02) — They are reached by different mechanisms — one by the issuer, one by the prober — and collapsing them is the defect this architecture exists to prevent.
  3. Verification does not share a failure domain with issuance (ADR-03) — Probers are deployed per region, outside the primary control plane, so an issuance outage does not also blind observation.
  4. Discovery is an input, not an audit feature (ADR-04) — A certificate the platform did not issue is still the platform's outage, so scanning and CT monitoring write into the same state machine as issuance.
  5. One control plane, two trust domains (ADR-05) — Public and private trust share a registry, an escalation ladder and a verify step. They share no protocol, authority, validation mechanism or lifetime, and the design does not pretend otherwise.
  6. Renewal at a fraction of life, not a fixed number of days (ADR-09) — One third remaining, so the policy survives the industry shortening maximum certificate lifetimes again — which it will.
  7. A failed renewal is an incident from the first attempt (ADR-10) — Escalation is measured against remaining validity, not against job status, which is what moves discovery from T-0 to T-30.
  8. Fail static (ADR-12) — An outage of this platform never invalidates anything already serving. Issuance queues, traffic continues, and the estate's survival time without the control plane is a published number.
  9. Trust before issuance, mechanically (ADR-13) — A new authority is distributed and confirmed everywhere before a leaf is signed beneath it, and a relying party that cannot report its bundle version blocks the rotation by name.
  10. Re-issue precedes revoke (ADR-15) — Revoking first turns a compromise notice into a self-inflicted outage. The order inverts only for a key known to be in an attacker's hands, and that inversion has an approver.
  11. Private keys never travel (ADR-18) — Keys are generated inside the consuming workload and only a CSR crosses the network, so the platform's databases hold no key material to lose.
  12. Authorisation is policy over attested identity (ADR-20) — Identity comes from a platform-attested credential, never a self-asserted subject, and every denial is recorded — a refusal nobody can investigate is not a control.

Why this should still be right in ten years

Certificate infrastructure outlives almost everything around it: a root signed today is still signing when the team that made it has gone, and the rules it has to obey are set by a standards body nobody in this organisation sits on. These are the properties that should survive a change of cloud, a change of CA, and another round of the industry shortening certificate lifetimes.

  • The truth rule names no technology. ADR-01 depends on nothing about DynamoDB, AWS, ACME or X.509. It is a statement about which of two sources to believe when they disagree, and it survives replacing every component underneath it. It is also the property a future team would most easily discard — by adding a fast path that marks a renewal complete on issuance — which is why SERVING is a state only the prober can write.
  • Renewal policy expressed as a fraction is future-proof by construction. Maximum public certificate lifetimes have fallen from years to months and are still falling. A policy that says "renew 30 days before expiry" quietly becomes a policy of renewing at half of life, then at all of it. A policy that says "one third of lifetime remaining" needs no change at all.
  • External deadlines only get tighter. The obligation to revoke and replace inside 24 hours is imposed from outside and has never been relaxed. Designing the bulk path as a rehearsed capability with a measured duration, rather than as an emergency, is the property that stops the next CA incident being an existential one.
  • Short lifetimes make the design more correct, not less. Every industry trend — shorter certificates, automated issuance, ubiquitous mTLS — increases issuance volume and decreases the value of revocation infrastructure. Both movements favour this architecture: lifetime becomes the control, and the renewal machinery is exercised continuously rather than discovered during an incident.
  • The verification plane is where the value accrues. As the estate grows, the registry's belief and reality drift further apart, and the cost of being wrong rises. The one component that would be tempting to cut in year one — the prober — is the one that becomes indispensable in year five.

Non-functional targets

Every number here is a stated assumption from the requirement, chosen to be argued with rather than believed. The right-hand column names the view where the mechanism that meets it is drawn.

Quality Target How it is met View
Certificates expiring while serving Zero, Sev-1 with a written review Expiry clock raised from the probe, not the registry; escalation ladder driven by remaining validity from the first failed attempt. 18
Private leaf issuance latency p95 ≤ 120 ms, p99 ≤ 250 ms in-region Attestation verified locally against a projected token; AWS Private CA called directly by the issuance plane with no control-plane hop on the hot path. 13
Public certificate, order to installed p95 ≤ 90 s, p99 ≤ 10 min Delegation already in place, so the challenge is answered inside the platform's own Route 53 zone; DNS propagation dominates the tail. 12
New domain to padlock ≤ 15 min p95, no support interaction Self-service CNAME delegation, named failure reasons on every validation outcome, issue before cutover. 04
Expiry detection delay ≤ 15 min p95 Regional probers sweep every known endpoint on a 15-minute cadence and compare the observed serial against the registry. 18
Renewal verified serving ≤ 10 min p95 after issuance Reload trigger inside the renewal transaction, followed by a targeted probe rather than the next scheduled sweep. 13
Revocation propagation ≤ 5 min p95, ≤ 15 min p99 CRL and OCSP published through CloudFront on a path independent of issuance; measured by a daily synthetic issue-install-revoke canary. 18
Bulk re-issuance of public trust ≤ 18 h, inside a 24 h mandate Rate-limit-aware bulk order queue across two pre-validated CA accounts; rehearsed twice a year against a representative subset. 15
Trust bundle distribution ≤ 4 h p95 to 100% of reporting parties Signed versioned bundle on its own lifecycle, with per-relying-party version reporting and a rotation gate that blocks below full coverage. 17
Private issuance availability ≥ 99.95% monthly Issuance plane across three AZ; a workload that cannot renew keeps serving until its leaf expires, which is the 8-hour margin. 16
CRL and OCSP availability ≥ 99.99% monthly Read-only, globally cached, published independently so that an issuance outage cannot prevent a revocation from being seen. 20
Estate survival without the control plane ≥ 8 h, published as a metric 24-hour leaf renewed at one third of life; fail static, so an outage stops change and stops nothing else. 16
Managed coverage of TLS endpoints ≥ 99.5%, remainder named and dated Network and DNS scanning plus CT monitoring reconciled against the registry; unmanaged findings carry an owner and a deadline. 09
Registry-to-reality divergence ≤ 0.1% at any probe cycle Divergence detector compares observed serial against registry state on every sweep and raises the gap as its own signal. 18
Registry recovery RPO 0, RTO ≤ 30 min DynamoDB global table; full rebuild from the append-only ledger plus one probe sweep, rehearsed annually. 10
Issuing intermediate replacement Live and distributed ≤ 4 h Two issuing intermediates kept live at all times, both already in every distributed trust bundle. 17
Root recovery RTO ≤ 72 h, human in the path by design Witnessed ceremony from hardware backup under two-person control; the one recovery path deliberately not automated. 16

Scope

In scope

  • A certificate registry spanning both trust domains, with an explicit lifecycle state model in which SERVING is reachable only by observation.
  • Public-trust issuance for customer custom domains over ACME, with DNS-01 CNAME delegation as the primary validation path and HTTP-01 as a stated-trade-off fallback.
  • Private PKI: an offline root, two live issuing intermediates, HSM custody, and short-lived workload leaves issued against attested platform identity.
  • Automated renewal at a fraction of lifetime remaining, with a jittered retry and an escalation ladder measured against remaining validity.
  • Delivery through a small set of supported adapters — mesh SDS, node agent, edge and ACM binding, secrets platform — with the reload trigger inside the renewal transaction.
  • Signed, versioned trust bundle distribution with per-relying-party version reporting and a mechanical gate ahead of issuance.
  • An independent verification plane: endpoint probing, network and DNS discovery, Certificate Transparency monitoring, divergence detection.
  • Revocation through CRL and OCSP on an independently published path, plus rehearsed bulk revocation and re-issuance against an external deadline.
  • An append-only issuance ledger, per-issuance attribution, ceremony records and the evidence exports compliance will ask for.

Explicitly out of scope

  • TLS termination itself. The platform supplies certificates to the edge and the mesh, and verifies them; it does not terminate.
  • The secrets platform, used as one delivery channel among several and owned elsewhere.
  • Code signing, document signing and any non-TLS use of the hierarchy.
  • Customer-supplied certificate upload, deferred to Phase 3 with its own renewal-reminder path.
  • Running the customer's DNS. The platform holds a delegated zone and answers challenges in it; the customer's registrar remains theirs.
  • The CMDB and service catalogue, which consume lifecycle events rather than being maintained here.

What a four-week prototype should prove

Four of this architecture's claims are cheap to test and expensive to be wrong about. A prototype that proves these can be built on; one that skips them is a demonstration of calling an ACME client, which is the easy half.

  1. The truth rule end to end: renew a certificate, deliberately suppress the reload, and confirm the platform holds the certificate at ISSUED, keeps it due for renewal, and escalates — rather than reporting success.
  2. Probe reach: instrument a representative slice of the estate and measure what fraction of TLS endpoints the prober can actually reach. The ≥ 99.5% coverage target is the number most likely to be wrong, and the unreachable remainder is the real blind spot.
  3. Delegation durability: onboard a domain with a CNAME delegation, then renew it three times across an artificially compressed schedule, confirming the tenant is never asked to act again and that a removed delegation is detected before it causes a failed renewal.
  4. Bulk capacity: time the re-issuance of a representative subset against both CA accounts under real rate limits, and extrapolate honestly. If 18 hours is not achievable, the architecture needs to know before a CA sets the clock.
  • Issued but not reloaded: renew a certificate, suppress the reload signal, and confirm the platform holds it at ISSUED, keeps it due, and escalates — rather than reporting a successful renewal.
  • Delegation withdrawn: remove a test tenant's _acme-challenge CNAME and confirm the delegation health check raises it long before the next renewal is due.
  • Control plane down: stop the control plane for four hours and confirm every workload keeps serving, orders queue rather than error, and the probers keep reporting.
  • Unreportable relying party: introduce a consumer that cannot report its trust bundle version and confirm the rotation gate blocks and names it, rather than proceeding.
  • Unmanaged certificate: stand up a TLS endpoint with a certificate the platform never issued and confirm discovery finds it, files it as UNMANAGED, and demands an owner.

Open risks, carried rather than hidden

Risk If it lands Response
The prober cannot reach enough of the estate If a material fraction of TLS endpoints is unprobeable, the truth rule degrades to belief for exactly the population most likely to be forgotten, and the headline guarantee weakens without any alarm firing. Measure coverage in the prototype. Unprobeable endpoints become a named architectural gap with an owner — a local agent, an exposed health surface, or an accepted exception — rather than an invisible remainder (ADR-03, ADR-04).
Short private leaves overrun the signing budget A 24-hour leaf at 3,800 workloads is 46,000 issuances a day before churn; a fleet-wide rollout is 37× steady state in ten minutes. Signing throughput is the binding constraint and it is not elastic. Size against measured AWS Private CA throughput early and treat leaf lifetime as the tuning dial. Lengthening the leaf relaxes the signing budget and lengthens the compromise window in the same move (ADR-08).
A relying party that cannot report its bundle version blocks rotation indefinitely The rotation gate is deliberately strict. A single legacy consumer with no reporting path can stall an authority rotation that has a deadline behind it. Inventory reporting capability before the first rotation, and treat an unreportable consumer as a migration item with a date. The escape hatch — an explicit, expiring, attributed exception — is a decision with an approver, never a flag (ADR-13).
Tenant churn leaves standing authority to issue A one-time CNAME delegation is the reason renewal is frictionless and also the reason the platform can issue for a customer's domain long after the relationship ends. Re-prove control on a declared cadence, check delegation health continuously, and retire on a schedule when a domain is repointed — with the contractual position stated rather than assumed (ADR-07).
Revocation is soft-fail in the clients that matter Most browsers do not hard-fail on an unreachable revocation check, so the propagation target can be met while the revocation has no practical effect. Declare a fail posture per relying-party class and treat an undeclared class as a defect. For workload identity, lean on lifetime rather than revocation, which is why the leaf is 24 hours (ADR-08, ADR-17).

The reasoning behind every component and technology choice is in the Architecture Decision Record: 20 records across 6 areas, each with the alternatives that lost and what the choice costs.