Hardening the wrong plane: ten years of Cloudflare's architecture
How one company's architecture changed between 2016 and 2026, and why rebuilding the data plane did not stop the outages that arrive through the configuration and machine-generated-data paths.
A single-company dig: Cloudflare, 2016 to 2026, reconstructed from eight of its own incident reports, its engineering accounts of Pingora, Quicksilver, Workers and the Rust rewrite of its core proxy, and the live argument in its open-source proxy's issue tracker. For architects who operate a uniform fleet and ship rules, flags, feature files or model artefacts to it, this names the three separate paths by which change reaches a server, shows that only one of them has a release protocol, and gives the validation, versioning and fail-open controls that the company's own remediation lists converge on.
The same oversized feature file hit both proxy generations on 18 November 2025 and the newer, memory-safe Rust one failed worse: FL2 returned 5xx where the old Lua proxy degraded to a bot score of zero, because an unwrap on a config-derived value turns a data error into an availability error.
What you get out of it
- Six of the eight published incidents here began with a change to configuration or machine-generated data; none began with a code release that passed review and then failed.
- There are three paths onto an edge server, not two: code, configuration, and a machine-generated data deploy that regenerates on a timer and has no version an operator can name.
- Memory safety is not failure safety. The pingora repository carries three independent reports of the panic-on-unwrap class in ten months, two still open, which is exactly the class that caused the November 2025 outage.
- After three configuration-propagation outages, propagation got faster: the 2026 successor to Quicksilver replicates a write to over 300 locations at a p99 of 256 ms. If you cannot slow propagation, every remaining control lives at the content boundary.
- The company already formally verifies configuration, with a purpose-built language and checker, for DNS only. Config verification applied to one subsystem is a capability, not a policy.
Scope
Why this, now. Two global outages seventeen days apart in late 2025, both caused by a configuration push rather than a code deploy, closed a ten-year pattern that is now documented well enough to read as a single arc.
What it does not cover. Cloudflare's security and Zero Trust products as products, its commercial performance, the Workers developer experience, and any comparison with other content delivery networks; claims about Oxy's internals, Quicksilver v2's storage engine and the current FL1 to FL2 traffic split stop where the public record does.
Other field guides
Replacing the package with the image: ten years of Red Hat changing the unit of change
Four generations of one host operating system, reconstructed from the artefacts the decisions were taken in: Fedora CoreOS design records and tracker…
30 sources · 10 organisations · 4 postmortemsEverything they deleted was on the inside: ten years of HashiCorp, read from its own repositories
HashiCorp spent a decade deleting things from the inside of its products: a Raft log store, a dependency cluster, a process on every node, four produ…
24 sources · 5 organisations · 1 postmortemKeeping the main branch green: thirteen years of merge queues, read from the repositories that ran them
The merge-queue pattern traced through its primary record: Rust's three generations of bors, Zuul's speculative gating, Kubernetes' Tide, GitLab's tr…
20 sources · 11 organisations · 2 postmortems