Keeping one failure from reaching everyone: a practitioner field guide

The territory

The problem, stated without the word "cell": how do you build a system in which one broken thing hurts some customers rather than all of them, and how do you know it worked before the day you need it to.

60s

Slack's time to drain traffic off one availability zone after siloing

Bethea, QCon SF 2024

73h

Roblox outage traced to one shared Consul cluster serving many workloads

Roblox postmortem, 2022

Nodes per cell in the EBS deployment of Physalia, chosen against correlated failure

NSDI '20, §2.2

Cost of data transfer inside one AZ, which is why AZ-shaped cells pay for themselves

AWS, 2021

The problem is old and the vocabulary is new. A system serves many customers from shared machinery; something in that machinery breaks; the question is what fraction of customers find out. Every organisation in this guide answered it the same structural way, by cutting the system into replicas that do not talk to each other and putting a thin routing layer in front, and every one of them invented a different word for the replica. AWS and Salesforce say cell. Shopify says pod. Slack says silo. Amazon's EBS configuration store, Physalia, says cell too, but it also says colony for the whole and color for something else entirely, and that second word is the most useful idea in the entire corpus.

What makes this topic worth a field guide right now is not the pattern, which is well documented and has been since AWS's re:Invent talk in 2019. It is the gap between the pattern and the incident record. Read AWS's Well-Architected guidance and you will come away believing the hard parts are choosing a partition key and sizing a cell. Read the five postmortems in section 4 and you will find that not one of them turned on either. Every one turned on a component that sat above the partitioning and touched everything: Cloudflare's Bot Management feature file, refreshed every few minutes and "published to the entire network"; DynamoDB's regional endpoint DNS record, which went empty and took 140-plus AWS services with it; Datadog's operating-system update channel, which fired inside the same one-hour window in five regions across three cloud providers.

The claim this guide makes

A production system has two planes that need partitioning, not one. The request plane carries user traffic and is what cells, pods and silos partition. The change plane carries code, configuration, policy, content, DNS and OS packages, and it is almost always still global. Cells bound the blast radius of a fault that arrives through the request plane. They do nothing whatsoever about a fault that arrives through the change plane, because the change plane reaches every cell by design. Physalia's authors wrote this down in 2020 and built a second, orthogonal partition for it. Almost nobody else has.

The wording matters here, so it is worth being precise about what is reported and what is reconstructed. That the five incidents propagated through shared change machinery is reported, in each organisation's own words. That this constitutes a general second axis of partitioning is my reconstruction, though it rests on a source that states it explicitly for one system: Physalia assigns every cell a colour, builds each cell only from nodes of one colour, and performs deployments "color-by-color", for the stated reason that operations "are dominant contributors to system downtime". The sources give this second axis four different names, none of them shared: colours (Physalia), waves (AWS Well-Architected), deployment rings (CrowdStrike's remediation), update channels (Datadog's root cause). In this guide I call it the change plane, and say so rather than pretending it is standard vocabulary.

Figure 1 · Two planes, one of which is usually unpartitioned

The request plane is cut into cells and every published architecture shows this. The change plane is drawn here as the sources describe it in their postmortems: a single pipeline that reaches every cell. Reconstructed from Cloudflare, Datadog and AWS.

Diagram source

Who has solved this in production, and how visibly. Slack migrated most user-facing services to per-AZ silos over roughly eighteen months and published both the design and the QCon talk about the organisational cost. Shopify has run podded MySQL since 2016 and published the routing component, Sorting Hat. Salesforce runs Hyperforce cells spanning three AZs each and publishes its deployment staggering. DoorDash built cells for cost first and isolation second. AWS runs cells in a dozen named services and publishes both a Well-Architected guidance document and a CDK reference implementation. Amazon's EBS team published the only peer-reviewed account, and it is the one that answers the questions the others skip.

What this guide does not cover. Multi-region active-active failover, which is a different problem with a different cost structure; database sharding as a scaling technique rather than an isolation technique; per-tenant single-tenant deployments, which are cells with a cell size of one and an economic profile this guide's sources do not describe; and the security case for tenant isolation, which overlaps but is driven by threat models rather than availability. It also does not cover Kubernetes multi-cluster management tooling, which the sources treat as an implementation detail rather than a decision.

How it is actually built

The shape that recurs across Slack, Shopify, Salesforce, DoorDash, AWS and Physalia, with the points where they genuinely diverge marked as such.

Lay the six published architectures side by side and four components appear in every column. A partition key that maps a request to exactly one cell. A router that performs the mapping and is deliberately kept stupid. Cells that contain a complete, independent copy of the workload including its data. And a drain control that can take a cell out of rotation faster than a human can be paged. The fifth component, a change plane partitioned in step with the cells, appears in exactly one column, which is the finding this guide is built around.

Figure 2 · Reference architecture, with the divergence points marked

Solid boxes appear in every published architecture. The dashed change pipeline is partitioned in step with the cells only in Physalia. Reconstructed from Slack, Shopify, Salesforce and Physalia.

Diagram source

The partition key, chosen against the grain

AWS states the rule plainly: keys "must be chosen to match the grain of the service, or the natural ways that a service's workload can be subdivided with minimal cross-grain interactions", and must be "easily accessible in most API calls". The worked warning is customer ID, which looks natural and breaks the moment one customer outgrows a cell. Shopify keys on shop_id; Slack keys on availability zone, which is unusual and deliberate; Physalia keys on the EBS volume's partition key, one cell per volume.

Runs this way at: AWS, Shopify

The router, deliberately stupid

AWS's re:Invent slide gives the rule in three words, "Thinnest possible layer", next to "Keep it simple!" The reasoning is that the router is the one component every request touches, so its availability caps the system's. Shopify's Sorting Hat matches a request to a pod and adds a header. AWS's own reference implementation goes further and returns the cell's DNS name at login, so the client caches it and talks to the cell directly; the router can then be down without the cells being down.

Runs this way at: AWS re:Invent, AWS reference impl

The cell, complete and unaware

"Fixed amount of resources / Well-defined function / No calling other cells / Will not expand / Diverse", per the re:Invent definition. The Well-Architected text is blunter about what breaks it: "Cross-cell dependencies can quickly eliminate the benefits of a cellular architecture". Note that Shopify's pods isolate only the datastores; app servers and job workers are shared, but "all shared resources can only ever communicate to a single pod at a time", which preserves the property that one request needs one pod.

Runs this way at: AWS, Shopify

The drain, which is the actual product

Slack's post is worth reading for this alone. Their cells exist so that a drain is possible: Envoy weights pushed by an in-house xDS control plane called Rotor, with "propagation through the control plane... on the order of seconds" and "a granularity of 1%". The target is to remove as much traffic as possible within five minutes; the talk reports 60 seconds achieved for siloed services. Without a fast drain, cells give you a smaller incident, not a shorter one.

Runs this way at: Slack

The health signal that decides

This is the least documented component and the one most likely to be wrong. Slack's triggering incident was a gray failure in which "systems within the impacted AZ saw complete availability of backends within their AZ, but backends outside the AZ were unavailable, and vice versa". A health check run from inside the failing zone reports healthy. The HotOS '17 definition is exact: gray failure is when "at least one app makes the observation that system is unhealthy, but observer observes that system is healthy". Your drain decision must be driven by client-observed success, not by the cell's opinion of itself.

Reported at: Slack; formalised in HotOS '17

The change plane, partitioned in exactly one system

Physalia assigns each cell a colour, builds each cell only from nodes of that colour, spreads colours evenly across the datacentre, and rolls out changes colour by colour, watching "for anomalies in single colors". The stated purpose is not deployment convenience: "Nodes of different colors don't communicate with each other, making it significantly less likely that a poison pill or overload could spread across colors." This is a second partition, orthogonal to the cell partition, on a different axis.

Runs this way at: Physalia (Amazon EBS)

Two divergence points are worth naming because they change the cost model. The first is what a cell contains. Salesforce's cell spans three availability zones and is redundant inside itself; DoorDash's cell is deliberately confined to one AZ so that traffic stays local and free. These are opposite answers to the same question and both are correct, because Salesforce is optimising for a cell surviving an AZ loss and DoorDash is optimising for the data-transfer bill. The second is how much is podded. Shopify pods only the stateful layer, on the stated grounds that databases are "the hardest component to scale"; Slack silos everything on the request path including stateless services, because the fault they were containing was a network fault that a stateless service is just as exposed to.

One reconstruction, flagged as such: none of the accounts explain how they test that a cell is genuinely independent. AWS's guidance says cross-cell dependencies eliminate the benefit, and Physalia's design goes to unusual lengths to avoid them, but no published source in this corpus describes an automated check that fails a build when a service in cell A acquires a dependency on cell B. Given how easily a shared cache, a shared feature flag service or a shared authentication endpoint creeps in, the absence of a published guard is itself a warning. If you build this, build the guard.

The decisions that matter

Six forks, each with what was chosen, what was rejected, the stated reason, and the condition under which the rejected option becomes the right one.

Decision: what is the cell boundary, an availability zone or a customer group?

Chosen

Slack and DoorDash: the availability zone.
The fault they were containing was infrastructural. Slack's June 2021 trigger was a faulty inter-AZ network link; DoorDash's driver was the cross-AZ data transfer bill.
It needs no application change to the data model.

Rejected

Partitioning by tenant, which Shopify and Salesforce chose instead.
For Slack it would not have helped: a bad AZ degrades every tenant, because a single API request "may fan out into hundreds of RPCs".

Flips when

Your dominant failure mode is a bad tenant rather than a bad zone: a query that melts a database, a customer whose growth outruns a shard.
You need per-customer blast radius for contractual reasons, which an AZ boundary cannot give you because every customer is in every AZ.

Decision: small cells or large cells?

Chosen

AWS's slide is explicit that this is a trade, not a preference. Smaller cells give "Reduced blast radius / Easier to test / Cells easier to operate".
Physalia went very small: seven nodes, one cell per EBS volume, millions of cells.

Rejected

Larger cells, which buy "Cost efficiency / Reduced splits / System easier to operate".
Physalia rejected the monolithic alternative knowing it costs complexity: "simplicity improves availability, so this simplification would be a boon", but the monolith "increases blast radius: a single bad software deployment could cause a complete failure".

Flips when

Your failures are uncorrelated and random rather than correlated. Physalia's Figure 4 makes the point: smaller cells are worse under a low rate of independent node failures and better once more than half the fleet is affected.
Per-cell fixed cost dominates, which is the usual case for a cell that carries a managed database with a minimum instance size.

Decision: should the router hold routing state, or should the client cache it?

Chosen

AWS's reference implementation authenticates at the router and then hands back the cell's DNS name, so the client talks to the cell directly.
The stated benefit is that the application keeps working "during a cell-router impairment".

Rejected

A router on the data path for every request, which is the obvious design and the one most teams build first.
Its availability becomes a ceiling on the whole system's, and it is now the biggest shared component you own.

Flips when

Cells move. Static client-side routing plus cell migration is how you get a client pinned to a cell that no longer holds its data. Physalia's answer is a discovery cache plus "not indefinite" forwarding pointers when cells move.
You need to drain in seconds. A cached endpoint cannot be drained; Slack's whole design depends on the router being live and weighted.

Decision: use the platform's zone affinity, or build routing yourself?

Chosen

Slack and DoorDash both built on Envoy with their own control planes: Rotor at Slack, a custom mesh control plane at DoorDash feeding locality into endpoint discovery.
This gives an explicit, observable weight per zone rather than a heuristic.

Rejected

Kubernetes Topology Aware Hints, whose own successor KEP says it "offers less user control and can be less predictable" and that "users filed issues reporting that hints weren't being applied or didn't function as expected".
Before that, the explicit topologyKeys field, deprecated in 1.21 and removed in 1.22.

Flips when

You want a cost optimisation rather than an isolation boundary. Kubernetes' trafficDistribution (GA in 1.33) is a good, cheap preference.
It never flips if you need a boundary: the documented behaviour is that hints are silently withheld if any node lacks a zone label or allocatable CPU, and that kube-proxy "falls back to using endpoints from all zones". A boundary that turns itself off is a preference.

Decision: deploy to all cells after a canary, or in waves?

Chosen

AWS Well-Architected states the rule: "deploy in waves, cell by cell or set of cells". Salesforce describes a "canary on a small set of cells, and then stagger the deployment across other cells".
Physalia goes further and makes the wave boundary a physical property of the cell (its colour) rather than a pipeline setting.

Rejected

One canary then everything, which is what AWS's own published reference implementation does: it "first updates the sandbox cell, checks it for aliveness and then updates all other cells".
That pipeline gives you a fleet-wide outage one aliveness check after a bad change.

Flips when

Never, for the production pipeline. It flips only for a change that is provably cell-local and reversible without a deploy.
Note the trap Physalia names: in a replicated system, "failure may not be evident until new code is deployed to half of all hosts", so a canary that watches only for crashes will pass a change that has already broken quorum tolerance.

Decision: does the fast-moving content path get the same treatment as code?

Chosen

After July 2024, CrowdStrike's answer is yes: content instances "are to be successively promoted to wider deployment rings or rolled back if problems are detected", with bake time between rings, plus "customer control over the deployment of Rapid Response Content".

Rejected

The design that existed before, in which content shipped through a channel deliberately built to move faster than the code release process.
Cloudflare's Bot Management feature file is the same shape: refreshed every few minutes, published to the entire network, and in November 2025 it took the network down.

Flips when

It does not flip on speed. It flips only if the content path is genuinely incapable of crashing the consumer, which means bounds-checked parsing and a defined behaviour for every malformed input, not a validator upstream.
Cloudflare's proximate cause was Result::unwrap() on an Err. A crash-free consumer would have degraded bot scoring instead of dropping traffic.

Figure 3 · Choosing the boundary

Every terminal node is an action, not a preference. Derived from the stated reasons in Slack's, Shopify's and AWS's accounts.

Diagram source

Summary of decisions, with the evidence each rests on.
Decision	Chosen	Rejected	Because	Evidence
Cell boundary	Availability zone (Slack, DoorDash)	Tenant group	The fault being contained was infrastructural, and fan-out spread it to every tenant anyway	Slack, 2023
Cell boundary	Tenant group (Shopify, Salesforce)	Availability zone	Isolation was needed against noisy neighbours and per-customer impact	Shopify
Cell size	Small, 7 nodes (Physalia)	One large store	A monolith "increases blast radius: a single bad software deployment could cause a complete failure"	NSDI '20
Router placement	Client caches the cell endpoint	Router on every request	Keeps cells reachable "during a cell-router impairment"	AWS reference impl
Zone routing	Envoy plus own control plane	Kubernetes topology hints	The hints heuristic is "less predictable" and silently disables itself	KEP-4444
Explicit topology API	Automatic hints, then `trafficDistribution`	`topologyKeys`	Deprecated in 1.21, removed in 1.22, replaced by automatic per-endpoint hints	k8s PR #96736
Deployment	Waves, cell by cell	Canary then all	"deploy in waves, cell by cell or set of cells"	AWS WA
Change-plane isolation	Colours, orthogonal to cells	Cells alone	Operations "are dominant contributors to system downtime"	NSDI '20 §3.4
Content path	Rings with bake time	Immediate global publish	"Template Instances should have staged deployment"	CrowdStrike RCA

What broke in production

Five published incidents, grouped by the class of boundary that failed. Note what is absent: no incident here is a cell failing and staying failed inside itself.

Three failure classes account for all five. Class A, the shared change artefact: one file, package or content blob generated centrally and delivered everywhere on a schedule (Cloudflare, Datadog, CrowdStrike). Class B, the shared routing state: the thin layer in front of the cells has its own state, and that state is global (AWS DynamoDB). Class C, the shared coordination substrate: one cluster that many workloads depend on, so its failure domain is the union of theirs (Roblox). Every one of them sits above the cell boundary, which is precisely why the cell boundary did not help.

Read them for the assumption rather than the mechanism. The mechanisms are all different and none will recur in your system. The assumptions are the same four every time, and all four are probably live in your architecture right now.

Postmortem

Class A · A feature file, refreshed every five minutes, published to the whole network

AssumptionConfiguration is not code, so it does not need the release process that code gets. The file had a size limit of 200 features, and the limit was treated as unreachable.

What happenedA ClickHouse permissions change at 11:05 UTC made table metadata visible that had not been before. The query generating the Bot Management feature file "does not filter for the database name", so it began returning duplicate rows and the file doubled in size. The proxy exceeded the 200-feature limit and "called Result::unwrap() on an Err value". Because the permissions rolled out gradually, "every five minutes there was a chance of either a good or a bad set of configuration files being generated", so the network recovered and failed repeatedly, which looked like an attack.

Blast radius11:20 UTC to 14:30 UTC for core traffic, 17:06 for full restoration. Core CDN, Turnstile, Workers KV, Access, Dashboard and Email Security. Cloudflare's own status page went down coincidentally, which is its own lesson.

FixHardening ingestion of internally generated configuration files, global kill switches for features, eliminating core-dump resource exhaustion, and a review of failure modes in every proxy module.

Design ruleAny artefact your fleet reloads on a timer is a deployment, and it needs waves, a bake time and a kill switch. If it is generated by a query, the query is production code and its result needs a schema check before publication, not after ingestion.

SourceCloudflare, 18 November 2025

Postmortem

Class A · Five regions on three clouds, one default update window

AssumptionRegions on different cloud providers are independent failure domains, because they share no network and no control plane.

What happenedA security update to systemd was applied automatically to VMs running Ubuntu 22.04. On restart, "systemd-networkd forcibly deleted the routes managed by the Container Network Interface (CNI) plugin (Cilium)". The interaction is only reachable on an already-running node: "Neither a fresh node nor a rebooted node exhibit this behavior", so no boot test would have found it. The regions failed together because "the time at which the automatic update happens is set by default in the OS to a window between 06:00 and 07:00 UTC".

Blast radiusTens of thousands of nodes lost connectivity between 06:00 and 07:00 UTC on 8 March 2023; roughly 50 to 60 percent of Kubernetes nodes. Full resolution 10 March at 06:25 UTC. The Pragmatic Engineer estimates about $5 million in lost revenue, roughly a day's revenue, with 500 to 600 engineers on the response.

FixDisabled the legacy security update channel across affected regions, changed systemd-networkd configuration to leave the routing table unchanged on restart, and audited the infrastructure for other legacy update channels.

Design ruleFailure domains are defined by shared inputs, not by shared infrastructure. Enumerate every channel that can change a machine without passing through your pipeline, including the base image, the package mirror and the vendor agent, and make each one wave-aware or turn it off.

SourceDatadog, March 2023; cost via The Pragmatic Engineer

Postmortem

Class A · A content channel built to bypass the release process, doing exactly that

AssumptionContent is validated upstream, therefore the consumer can trust its shape. A template type declared 21 input fields; the code that invoked it supplied 20.

What happenedThe mismatch was invisible for months because test instances used a wildcard for the 21st field. On 19 July 2024 an instance shipped with a non-wildcard criterion on that field, the Content Interpreter read the 21st entry of a 20-entry array, and the sensor crashed. Two independent gaps were needed: "The number of fields in the IPC Template Type was not validated at sensor compile time" and "A runtime array bounds check was missing".

Blast radiusGlobal, immediate, and on hosts the vendor did not own. Recovery required physical or console access to many machines, which is the worst possible property for a change-plane failure.

FixBounds checking in the Content Interpreter, a validator change permitting only wildcards in the 21st field, staged deployment with "successively promoted... wider deployment rings" and bake time, and, notably, "customer control over the deployment of Rapid Response Content".

Design rulePush the boundary into the consumer. A validator upstream of a global fan-out protects nothing once it has a logic error, because there is no second opinion downstream. The consumer must survive malformed input from its own vendor, and the customer must be able to choose their ring.

SourceCrowdStrike RCA, 6 August 2024

Postmortem

Class B · The router had state, and the state went empty

AssumptionRedundancy in the automation that maintains routing is equivalent to isolation. DynamoDB's DNS Enactor runs redundantly across three availability zones.

What happenedThree actors, one piece of shared state. A delayed Enactor applied "a much older plan", overwriting a newer one; the cleanup process then deleted that older plan, "removing all IP addresses for the regional endpoint" for dynamodb.us-east-1.amazonaws.com, and left the system in a state its own automation could not repair. Then the cascade: EC2's DropletWorkflow Manager depends on DynamoDB for droplet leases, and when DynamoDB returned, "DWFM had entered a state of congestive collapse and was unable to make forward progress".

Blast radius19 October 2025, 23:48 PDT to 20 October, 13:50 PDT. DNS restored at 02:25, but EC2 throttling did not lift until 13:50 and NLB health-check failover was manually disabled between 09:36 and 14:09. Over 140 AWS services affected.

FixThe DNS Planner and Enactor automation was disabled worldwide pending a race-condition fix, plus protections against applying stale plans, NLB velocity controls limiting how fast capacity can be removed on health-check failure, and EC2 scale tests that exercise DWFM recovery.

Design ruleThe routing layer is not exempt from cellularisation just because it is thin. Any writer of shared routing state needs a generation check that cannot go stale, and an "empty result is never valid" invariant. Separately: test the recovery path at full fleet scale, because congestive collapse on recovery is a different failure from the one you drilled.

SourceAWS, October 2025

Postmortem

Class C · One coordination cluster, and the telemetry inside it

AssumptionA service-discovery cluster is infrastructure, so consolidating workloads onto one is efficiency rather than coupling.

What happenedA newly enabled Consul streaming feature under high read and write load caused contention on a single Go channel, and the load pattern separately triggered a pathological free-list cost in BoltDB. Roblox's own summary: "A single Consul cluster supporting multiple workloads exacerbated the impact of these issues." Diagnosis was crippled by "a circular dependency between our telemetry systems and Consul", so the failure erased the evidence of itself.

Blast radius73 hours, 28 to 31 October 2021, essentially the whole platform. Almost all of it was diagnosis time rather than repair time.

FixAn additional geographically distinct data centre for backend services, and work to move to multiple availability zones within data centres. In other words, the remediation was to build the cell boundary that did not exist.

Design ruleEvery shared coordination service is a cell boundary you have not drawn. Partition it per cell even where that is wasteful, and put the observability for a cell strictly outside it, because a detector inside the failure domain converts an outage into a mystery.

SourceRoblox, January 2022

Paper

The failure class with no postmortem: gray failure inside a healthy-looking cell

AssumptionA cell that reports healthy is healthy, so automated drain can be driven by the cell's own health check.

What happenedSlack's June 2021 trigger is the concrete instance: a faulty inter-AZ link meant "systems within the impacted AZ saw complete availability of backends within their AZ, but backends outside the AZ were unavailable, and vice versa". HotOS '17 gives the general form: gray failure is when "at least one app makes the observation that system is unhealthy, but observer observes that system is healthy". The same paper shows why fan-out amplifies it: with fan-out m and n redundant components, the chance a request touches any given one is 1-((n-1)/n)^m, which "rapidly approaches 100% as m becomes large", so "increasing redundancy can counter-intuitively hurt availability".

Blast radiusNot published as an incident report. Slack describes user-visible degradation across the platform from a fault confined to one AZ, which is the point: the blast radius was the whole system despite the fault being local.

FixSlack's silos plus a drain driven by observed traffic success, with 1% weight granularity, rather than by a binary health verdict.

Design ruleHealth checks must be evaluated from outside the cell, by the caller, on real request outcomes. If your drain trigger and your failing component share a vantage point, you have built a detector that agrees with the fault.

SourceHuang et al., HotOS '17; instance from Slack

Figure 4 · How a change-plane fault reaches every cell, in order

The generic sequence behind Cloudflare's and Datadog's incidents. Notice that the cell boundary is never crossed: the fault is delivered inside each cell independently, at the same moment. Reconstructed from Cloudflare's and Datadog's postmortems.

Diagram source

The absence worth noticing

Not one incident in this corpus is "a cell failed and its customers were affected while everyone else was fine". That is not proof cells work. A contained failure is, by construction, not newsworthy enough for a public postmortem, so the published record is structurally biased against exactly the evidence you want. The practical consequence: you cannot learn that your containment works from anybody else's writing. You learn it by draining a production cell on a Tuesday and measuring who noticed.

Numbers you can plan against

Everything quantitative in the corpus, with the context it was measured in and the date it was true.

M = measured by the source. C = claimed without published method. D = derived here, arithmetic shown. U = nobody has published it.
Metric	Value	At	Context	As of	Kind	Source
Cell drain time, siloed services	60 s	Slack	Weighted Envoy drain of one AZ; target was 5 min	2024	M	QCon SF
Drain weight granularity	1%	Slack	Rotor xDS control plane; propagation "on the order of seconds"	2023	M	Slack
Migration duration, critical services	>1 yr	Slack	Per-service, bottom-up adoption after a top-down attempt failed	2024	M	QCon SF
Nodes per cell	7	Physalia (EBS)	Paxos group; durability "around 5000x higher" than 2-replication	2020	M	NSDI '20
Cells per datacentre installation	millions	Physalia (EBS)	One cell per EBS volume partition key; over 60 datacentre-scale deployments	2020	M	NSDI '20
Cross-AZ data transfer	charged	AWS	Same-AZ transfer is free; cross-AZ is billed, which is the economics behind AZ-shaped cells	2021	M	AWS
Topology hints overload ceiling	20%	Kubernetes	Hints are not added unless expected initial overload is below this	2026	M	KEP-2433
Topology hints redistribution threshold	30%	Kubernetes	Above this the controller redistributes, and removes hints if it cannot	2026	M	KEP-2433
Bot Management feature limit	200	Cloudflare	Runtime cap; exceeding it panicked the proxy	2025	M	Cloudflare
Config republish interval	~5 min	Cloudflare	Feature file regenerated and published network-wide on this cycle	2025	M	Cloudflare
Outage, core traffic	3 h 10 m	Cloudflare	11:20 to 14:30 UTC; full restoration 17:06	2025-11	M	Cloudflare
Outage, DNS repair vs full recovery	2.6 h vs 14 h	AWS us-east-1	DNS restored 02:25 PDT; EC2 throttling lifted 13:50 PDT	2025-10	D	AWS
AWS services affected	140+	AWS us-east-1	Services depending on DynamoDB, EC2 or NLB in the region	2025-10	C	AWS
Nodes losing connectivity	50-60%	Datadog	Kubernetes nodes in production, tens of thousands, in one hour	2023-03	M	Datadog
Outage revenue impact	~$5 M	Datadog	Estimated as about one day of usage-based revenue, not a company figure	2023-05	C	Pragmatic Engineer
Engineers on the response	500-600	Datadog	Across a 24-hour-plus incident on five regions	2023-05	C	Pragmatic Engineer
Outage duration	73 h	Roblox	Mostly diagnosis, not repair; telemetry depended on the failed component	2021-10	M	Roblox
AZs per cell	3	Salesforce Hyperforce	Cell spans AZs for redundancy inside the cell; target 99.99%	2022-08	M	Salesforce
Cost multiplier of running cells	unpublished	everyone	Slack says double-running the site during migration was unaffordable; no one publishes a steady-state figure	2026	U	QCon SF

What to actually plan against. Three of these numbers transfer and the rest are context. The first is the drain target: Slack aimed for five minutes and achieved 60 seconds, so if your runbook for removing a failure domain takes longer than about five minutes, the cells are not yet buying you anything an incident commander can use. The second is the ratio in the AWS row: the fault was repaired in 2.6 hours and the system recovered in 14. Budget recovery time at several multiples of repair time, and test recovery at full fleet scale, because congestive collapse only appears there. The third is the pair of Kubernetes thresholds, 20% and 30%, which are worth internalising for a different reason: they tell you the platform's zone affinity is designed to switch itself off rather than overload a zone. That is the right default for a cost optimisation and the wrong one for a containment boundary.

Read these carefully

The $5 M and 500-600 engineer figures are a third party's estimate and a reported number, not Datadog's accounting. The "140+ services" figure is AWS's characterisation of its own event. The recovery-versus-repair ratio in the AWS row is derived here by subtracting published timestamps, and the arithmetic is 02:25 minus 23:48 for repair and 13:50 minus 23:48 for recovery. Every cost figure in this corpus dates from 2021 to 2025 and cloud pricing moves; re-check the transfer rate against the current pricing page before you put it in a business case.

The evidence wall

Twenty-eight artefacts across seventeen hosts and sixteen organisations, graded. Filter by kind; the postmortems and the source code are where the value is.

Postmortem AWS2025-10

Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region

The most detailed public account of a routing-layer failure. Two redundant automation actors racing on shared DNS state produced an empty regional endpoint record, then EC2's lease manager could not converge on recovery.

Carry forwardAny writer of shared routing state needs a generation check that cannot go stale, and an invariant that an empty result is never publishable. Test recovery at full fleet scale.

https://aws.amazon.com/message/101925/

Postmortem Cloudflare2025-11

Cloudflare outage on November 18, 2025

A database permissions change, an unfiltered query, a doubled feature file, a 200-feature limit and an unchecked unwrap(). Includes the internal chat showing the team reasonably suspecting a DDoS, because gradual upstream rollout made it flap.

Carry forwardAnything the fleet reloads on a timer is a deployment. Give it waves, a kill switch and a consumer that cannot panic.

https://blog.cloudflare.com/18-november-2025-outage/

Postmortem Datadog2023-03

2023-03-08 Incident: Infrastructure connectivity issue affecting multiple regions

The cleanest published example of independent regions failing together through a shared input. Also unusually honest about testability: the interaction was unreachable on a freshly booted node, so no pre-production test could have found it.

Carry forwardEnumerate every channel that can change a machine without passing through your pipeline. Each is a failure domain you did not draw.

https://www.datadoghq.com/blog/2023-03-08-multiregion-infrastructure-connectivity-issue/

Postmortem Roblox2022-01

Roblox Return to Service 10/28-10/31 2021

Seventy-three hours, almost all of it diagnosis. The two named contributing structures are a single Consul cluster serving many workloads and a circular dependency between telemetry and that cluster.

Carry forwardPut a cell's observability strictly outside the cell. A detector inside the failure domain turns an outage into a mystery.

https://about.roblox.com/newsroom/2022/01/roblox-return-to-service-10-28-10-31-2021

Postmortem CrowdStrike2024-08

Channel File 291 Incident Root Cause Analysis

A 21-versus-20 field mismatch that survived months of testing because every test used a wildcard on the field that differed. The remediation list is the best published statement of what a change plane needs: rings, bake time and customer-controlled rollout.

Carry forwardAn upstream validator with a logic error protects nothing. The consumer must survive malformed input from its own vendor.

https://www.crowdstrike.com/wp-content/uploads/2024/08/Channel-File-291-Incident-Root-Cause-Analysis-08.06.2024.pdf

Paper Amazon / USENIX NSDI '202020-02

Millions of Tiny Databases (Physalia)

The only peer-reviewed account of a cellular system in this corpus, and the only source that treats the change plane as a first-class partitioning problem. Section 3.4 introduces colours; section 3.3 explains why replication weakens canarying.

Carry forwardCells partition the request path. A second, orthogonal partition is needed for deployments, patching and load, and it must be a property of the node, not a setting in the pipeline.

https://www.usenix.org/conference/nsdi20/presentation/brooker

Paper Microsoft Research, HotOS '172017-05

Gray Failure: The Achilles' Heel of Cloud-Scale Systems

Defines the failure mode that defeats automated containment, and proves the counter-intuitive corollary that under high fan-out, more redundancy means a higher probability that any given request touches the degraded component.

Carry forwardDrain decisions must be driven by caller-observed success, never by the failing component's own health check.

https://www.microsoft.com/en-us/research/wp-content/uploads/2017/06/paper-1.pdf

Talk AWS re:Invent 20192019-12

ARC411-R1: Reducing blast radius with cell-based architectures

The canonical five-property definition of a cell and the three-word router rule. Also the clearest statement that cell size is a genuine trade rather than a preference, with the benefits of large cells stated as honestly as those of small ones.

Carry forward"Thinnest possible layer." If your router grows business logic, it has become the largest shared component you own.

https://d1.awsstatic.com/events/reinvent/2019/REPEAT_1_Reducing_blast_radius_with_cell-based_architectures_ARC411-R1.pdf

Talk Slack / QCon SF2024-11

Slack's Migration to a Cellular Architecture (Cooper Bethea)

The organisational account the blog post leaves out: the first, top-down design was abandoned because running two production environments was unaffordable, and adoption only worked bottom-up, service by service, over more than a year for the critical ones.

Carry forwardBudget the migration as a multi-year programme with a per-service opt-in, and measure progress by drain time, weekly.

https://www.infoq.com/presentations/slack-cellular-architecture/

Eng blog Slack2023-08

Slack's Migration to a Cellular Architecture

The best public description of siloing by availability zone, of the drain machinery, and of the gray failure that triggered it on 30 June 2021. Ends on the practical obstacle nobody mentions: legacy discovery interfaces, and DNS in particular.

Carry forwardInventory your service-discovery interfaces before designing the drain. Anything that resolves through DNS cannot be partially drained.

https://slack.engineering/slacks-migration-to-a-cellular-architecture/

Eng blog Shopify2019

A Pods Architecture To Allow Shopify To Scale

The clearest account of the distinction between sharding and isolating. Sharding in 2015 solved capacity and left a platform-wide failure mode; pods in 2016 made a request need exactly one shard. Only the datastores are podded.

Carry forwardSharding is not isolation. The property that matters is that serving a request requires exactly one partition to be online.

https://shopify.engineering/a-pods-architecture-to-allow-shopify-to-scale

Eng blog Salesforce2022-08

Architectural Principles for High Availability on Hyperforce

Shows cells and availability zones composed rather than substituted: a cell spans three AZs and is internally redundant. Also states the deployment staggering explicitly, canary on a small set of cells then stagger across the rest.

Carry forwardA cell is a unit of blast radius, not a unit of redundancy. It still needs redundancy inside it.

https://engineering.salesforce.com/architectural-principles-for-high-availability-on-hyperforce/

Eng blog InfoQ, reporting DoorDash2024-01

DoorDash Uses Service Mesh and Cell-Based Architecture to Significantly Reduce Data Transfer Costs

The cost-first case for cells: one Kubernetes cluster per cell, no inter-cell traffic, and Envoy zone-aware routing keeping calls local. Second-hand, because DoorDash's own post returned HTTP 403 during this research; InfoQ quotes their engineers directly.

Carry forwardAZ-shaped cells can be cost-positive, which is the strongest argument you will have in a funding conversation. Zone affinity must hold at every hop or the saving evaporates.

https://www.infoq.com/news/2024/01/doordash-service-mesh/

Eng blog The Pragmatic Engineer2023-05

Inside Datadog's $5M Outage

Independent analysis of the March 2023 incident with the cost estimate and the size of the response. Useful mainly for the framing that multi-cloud bought Datadog nothing, because the shared thing was the operating-system image.

Carry forward"Multi-cloud" is not a failure-domain claim. Ask what artefact is identical across the clouds.

https://newsletter.pragmaticengineer.com/p/inside-the-datadog-outage

Decision record Kubernetes SIG Network2021

KEP-2433: Topology Aware Hints

The design record for the platform's zone affinity, and it is refreshingly explicit that the whole thing rests on an assumption: traffic is proportional to allocatable CPU per zone. States the 20% and 30% thresholds and the kube-proxy fallback.

Carry forwardRead the assumptions section before adopting. If your traffic does not originate proportionally to capacity, the heuristic is wrong for you and will either overload a zone or switch itself off.

https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/2433-topology-aware-hints/README.md

Decision record Kubernetes SIG Network2024-2026

KEP-4444: Traffic Distribution for Services

The record of the heuristic being superseded, and the reason given is operability rather than correctness: less user control, less predictable, and users reporting that hints were not applied. Alpha in 1.30, GA in 1.33.

Carry forwardTwo successive redesigns of the same feature in five years is the signal. Do not build an isolation boundary on top of it.

https://github.com/kubernetes/enhancements/blob/master/keps/sig-network/4444-service-traffic-distribution/README.md

Source kubernetes/kubernetes2020-12

PR #96736: core/v1: deprecate Service topologyKeys

The recorded rejection of the explicit approach. topologyKeys let an operator state a preference order directly; it was deprecated in 1.21 and removed in 1.22 in favour of automatic per-endpoint hints.

Carry forwardThe platform deliberately chose an advisory, automatic mechanism over an explicit one. If you need explicit, you are building it yourself, as Slack and DoorDash both did.

https://github.com/kubernetes/kubernetes/pull/96736

Source kubernetes/kubernetes2022-11

Issue #113731: Support topology aware routing (node and zone)

The request to bring back deterministic rules after topologyKeys was removed, and the most direct statement of what is wrong with the replacement: "Sometimes the CPU-balance heuristic is wrong... we may erroneously disable hints", plus "Hints do not work at low-endpoint counts" and "Hints are not deterministic".

Carry forwardDeterminism is the property an isolation boundary needs and the property the heuristic explicitly does not offer. This issue is the bridge between the 2020 removal and the 2024 replacement.

https://github.com/kubernetes/kubernetes/issues/113731

Source envoyproxy/envoy2023-07

Issue #28419: Zone-aware load balancing incorrectly handles mismatched local and upstream localities

Three separate bugs, all triggered by the same real-world condition: the caller's zone set and the callee's zone set are not identical. Sorting collapsed empty localities and misaligned the index mapping, so interchangeable zones got different traffic. Fixed in Envoy 1.28.0, October 2023.

Carry forwardZone affinity assumes symmetry you will not have. Pin the Envoy version, and assert zone distribution in a test rather than trusting it.

https://github.com/envoyproxy/envoy/issues/28419

Source envoyproxy/envoy2026-02

Issue #43665: Proposal, ORCA-driven locality routing

The still-live limitation, stated by the proposer: zone-aware routing splits traffic on healthy host counts or weights and "has no visibility into actual backend load", so uneven demand produces hotspots the router cannot see.

Carry forwardZone affinity balances by capacity, not by load. If your demand is skewed by zone, you need per-cell load signal in the routing decision.

https://github.com/envoyproxy/envoy/issues/43665

Source istio/istio2019-10

Issue #18500: support zone aware routing of envoy

Opened in October 2019 and closed without Envoy's zone-aware routing being exposed. The reported problem with the mesh's alternative is concrete: locality weighted load balancing imbalances "if replicas of a service is NOT equal in each zones", which is the normal state of a Kubernetes deployment.

Carry forwardA feature request that stays open for years is a statement about the maintainers' model of the problem. Do not plan around it landing.

https://github.com/istio/istio/issues/18500

Source AWS Solutions Library2023-2026

guidance-for-cell-based-architecture-on-aws

A working CDK implementation of router plus cells, useful for the routing pattern where the client caches the cell endpoint. Worth reading against the vendor's own guidance: the pipeline updates a sandbox cell, checks aliveness, then updates all other cells.

Carry forwardRead the reference implementation's pipeline, not just its architecture. One canary then everything is the change-plane anti-pattern this guide is about, shipped in the vendor's own sample.

https://github.com/aws-solutions-library-samples/guidance-for-cell-based-architecture-on-aws

Vendor AWS Well-Architected2023-09

Reducing the Scope of Impact with Cell-Based Architecture: Cell design

The ideal stated plainly, "no cross-cell API calls, no shared resources like databases or S3 buckets", along with the admission that it is not always achievable and that cross-cell dependencies "can quickly eliminate the benefits".

Carry forwardMake cross-cell dependency a build-time assertion, not a review-time aspiration. No source in this corpus publishes such a check.

https://docs.aws.amazon.com/wellarchitected/latest/reducing-scope-of-impact-with-cell-based-architecture/cell-design.html

Vendor AWS Well-Architected2023-09

Cell partition

The partition-key guidance, including the grain rule and the worked warning about customer ID failing when one customer outgrows a cell. Recommends routing unavoidable cross-cell calls back through the router rather than cell to cell.

Carry forwardDesign the composite key on day one. Retrofitting a second partition dimension after a customer outgrows a cell is a migration, not a config change.

https://docs.aws.amazon.com/wellarchitected/latest/reducing-scope-of-impact-with-cell-based-architecture/cell-partition.html

Vendor AWS Well-Architected2023-09

Cell deployment

States the wave rule, "deploy in waves, cell by cell or set of cells", and is unusually honest about the operational bill: you now have "tens, hundreds or even thousands of instances of your workload to deploy and operate".

Carry forwardThe pipeline is a prerequisite, not a follow-up. Cells without wave deployment are a bigger fleet with the same blast radius.

https://docs.aws.amazon.com/wellarchitected/latest/reducing-scope-of-impact-with-cell-based-architecture/cell-deployment.html

Vendor AWS Well-Architected2023-09

Reducing the Scope of Impact with Cell-Based Architecture (overview)

Positions cells as bringing "the same fault isolation concepts that AWS applies in its Availability Zones and Regions to the level of your workload architecture", and notes AWS service teams have used the pattern for more than a decade.

Carry forwardUseful framing for a design review: you are buying the isolation property the cloud provider sells you, one layer up.

https://docs.aws.amazon.com/wellarchitected/latest/reducing-scope-of-impact-with-cell-based-architecture/reducing-scope-of-impact-with-cell-based-architecture.html

Vendor Kubernetes2026

Topology Aware Routing (documentation)

The five documented conditions under which hints are silently withheld, including a node missing a zone label or an allocatable CPU value, and the caution that the approach "will not work well for Services that have a large proportion of traffic originating from a subset of zones".

Carry forwardAlert on hints being absent. Silent fallback to cross-zone routing is both a cost regression and a loss of the boundary you thought you had.

https://kubernetes.io/docs/concepts/services-networking/topology-aware-routing

Vendor AWS Architecture Blog2021-06

Overview of Data Transfer Costs for Common Architectures

The pricing asymmetry that makes AZ-shaped cells pay for themselves: transfer within an availability zone is free, and transfer across them is charged. This is the economic engine behind DoorDash's design.

Carry forwardModel the transfer bill before the resilience case. In a chatty microservice estate, the cost argument for zonal cells is often the stronger one.

https://aws.amazon.com/blogs/architecture/overview-of-data-transfer-costs-for-common-architectures/

On the shape of this evidence

Five postmortems, two papers, two talks, five engineering blogs, six source references, two decision records and six vendor documents, across sixteen organisations. Two honest weaknesses. First, DoorDash's own post was unreachable during this research (HTTP 403), so those claims are carried second-hand through InfoQ and are marked as such. Second, AWS appears in five roles here, as vendor, as talk, as paper author, as source and as the subject of a postmortem, which means a single organisation's engineering culture is doing a lot of work in this synthesis. Where AWS is the only source for a claim, the text says so.

Build a miniature, then productionise it

Six rungs. The line from toy to production-shaped is crossed at rung four, where you stop testing the cells and start testing the change plane.

Two cells, one router, one partition key

Stand up the same small service twice, backed by two separate datastores. Put a router in front that hashes a tenant id to a cell and nothing else. Resist every temptation to put logic in the router.

Done when: killing cell B's datastore leaves every cell A tenant working, with no errors in cell A's logs. Teaches: that isolation is a property of the request path, not of the deployment topology.

Move the routing decision to the client

Change the router so that it returns the cell's endpoint at session start and the client then talks to the cell directly, as AWS's reference implementation does. Then kill the router.

Done when: existing sessions continue working with the router down, and only new sessions fail. Teaches: the difference between a control plane and a data plane, felt rather than read.

Build the drain before you need it

Add per-cell weights, pushed from a control plane, with 1% granularity, as Slack does. Wire a single command that takes a cell to zero. Time it.

Done when: a cell goes from 100% to 0% of traffic in under 60 seconds, measured at the client, and you have the graph. Teaches: that the drain, not the cell, is what an incident commander actually uses.

Break the health check on purpose

Inject a gray failure: make cell B slow rather than dead, or make it healthy to its own probe and failing to callers. Then see whether your automation drains it.

Done when: the drain fires on caller-observed success rate, and you can show it does not fire on the cell's self-report. Teaches: differential observability, the reason Slack's June 2021 fault degraded everything.

Partition the change plane

This is the crossing. Give each cell a wave label. Make the pipeline deploy wave by wave with a bake time between waves, and make it refuse to deploy to two waves at once. Then find every other path that can change a machine: base images, package mirrors, configuration pushed on a timer, feature flags, vendor agents, and put each behind the same waves or delete it.

Done when: you have a written inventory of change channels, each one either wave-aware or disabled, and a deliberately poisoned config artefact stops at wave one. Teaches: that Cloudflare, Datadog and CrowdStrike were all one inventory away from a much smaller incident.

Put the observability outside, and drill in production

Move the metrics, logs and alerting for a cell out of that cell, so a dead cell still reports. Then run a monthly production drain of one real cell during business hours, with a stopwatch and a named commander.

Done when: a production cell has been drained and restored without a customer-visible error budget burn, three months running, and the drain time is trending down. Teaches: the only evidence that will ever exist that your containment works, since nobody else's postmortem contains it.

Keep hunting

The queries that actually produced the material above. The vocabulary is the useful part: each organisation names the same idea differently, so searching one name finds one company.

Production accounts, by the word each org uses

slack engineering cellular architecture availability zone "we" migration
shopify pods podded architecture engineering sharding tenants
salesforce OR canva OR atlassian engineering "cell" architecture multi-tenant isolation "we"
"cell-based architecture" blast radius "we" production lessons learned

The change plane, which is where the incidents are

postmortem "all regions" OR "every region" simultaneously "automatic update"
outage "config file" OR "feature file" "pushed to" OR "published to the entire"
"staged deployment" OR "deployment rings" remediation root cause analysis 2024..2026
incident "single cluster" OR "shared cluster" "multiple workloads" root cause

Decision records and rejected designs

site:github.com kubernetes enhancements keps sig-network topology aware hints README
github "deprecate" service topologyKeys "will be replaced"
envoy zone aware routing github issue "does not work" locality weighted problem
repo:envoyproxy/envoy is:issue "zone aware" sort:reactions-+1-desc

The vocabulary to chain from

"poison pill" transaction distributed state machine deployment colors
"gray failure" "differential observability" availability zone drain
"thinnest possible layer" cell router control plane data plane
"shuffle sharding" OR "cell router" OR "colony" blast radius paper

Two things about this hunt worth passing on. The single highest yield move was reading the Physalia paper's section 3.4 and then searching the term it taught, colors, alongside poison pill; that is what turned a survey of cell architectures into an argument about the change plane. The second is that searching for the pattern's name mostly returns explanations of the pattern. Searching for the failure, in the vocabulary of the postmortem rather than the vocabulary of the architecture, is what returns experience.

References

AWS, Summary of the Amazon DynamoDB Service Disruption in the Northern Virginia (US-EAST-1) Region Amazon Web Services, October 2025. Checked 2026-08-29.
Cloudflare, Cloudflare outage on November 18, 2025 Cloudflare blog, 18 November 2025. Checked 2026-08-29.
Datadog, 2023-03-08 Incident: Infrastructure connectivity issue affecting multiple regions Datadog, March 2023. Checked 2026-08-29.
Roblox, Roblox Return to Service 10/28-10/31 2021 Roblox Newsroom, January 2022. Checked 2026-08-29.
CrowdStrike, Channel File 291 Incident Root Cause Analysis CrowdStrike, 6 August 2024. Checked 2026-08-29.
Brooker, Chen and Ping, Millions of Tiny Databases USENIX NSDI '20, February 2020. Checked 2026-08-29.
Brooker, Chen and Ping, Millions of Tiny Databases (full text PDF) Amazon Science mirror, 2020. Checked 2026-08-29.
Huang, Guo, Zhou, Lorch, Dang, Chintalapati and Yao, Gray Failure: The Achilles' Heel of Cloud-Scale Systems ACM HotOS '17, May 2017. Checked 2026-08-29.
AWS, ARC411-R1: Reducing blast radius with cell-based architectures AWS re:Invent, December 2019. Checked 2026-08-29.
Cooper Bethea, Slack's Migration to a Cellular Architecture QCon San Francisco 2024, published on InfoQ 2 May 2025. Checked 2026-08-29.
Slack, Slack's Migration to a Cellular Architecture Slack Engineering, August 2023. Checked 2026-08-29.
Shopify, A Pods Architecture To Allow Shopify To Scale Shopify Engineering. Checked 2026-08-29.
Salesforce, Architectural Principles for High Availability on Hyperforce Salesforce Engineering, 10 August 2022. Checked 2026-08-29.
InfoQ, DoorDash Uses Service Mesh and Cell-Based Architecture to Significantly Reduce Data Transfer Costs InfoQ, 23 January 2024. Checked 2026-08-29.
Gergely Orosz, Inside Datadog's $5M Outage The Pragmatic Engineer, 16 May 2023. Checked 2026-08-29.
Kubernetes SIG Network, KEP-2433: Topology Aware Hints kubernetes/enhancements. Checked 2026-08-29.
Kubernetes SIG Network, KEP-4444: Traffic Distribution for Services kubernetes/enhancements. Checked 2026-08-29.
Andrew Sy Kim, core/v1: deprecate Service topologyKeys (PR #96736) kubernetes/kubernetes, merged 9 December 2020. Checked 2026-08-29.
Kubernetes, Issue #113731: Support topology aware routing (node and zone) kubernetes/kubernetes, opened 8 November 2022. Checked 2026-08-29.
Envoy, Issue #28419: Zone-aware load balancing incorrectly handles mismatched local and upstream localities envoyproxy/envoy, opened 16 July 2023. Checked 2026-08-29.
Envoy, Issue #43665: Proposal, ORCA-driven locality routing for zone-aware load balancing envoyproxy/envoy, opened 26 February 2026. Checked 2026-08-29.
Istio, Issue #18500: support zone aware routing of envoy istio/istio, opened 31 October 2019. Checked 2026-08-29.
AWS Solutions Library, Guidance for Cell-Based Architecture on AWS GitHub reference implementation. Checked 2026-08-29.
AWS, Reducing the Scope of Impact with Cell-Based Architecture AWS Well-Architected, 20 September 2023. Checked 2026-08-29.
AWS, Cell design AWS Well-Architected, September 2023. Checked 2026-08-29.
AWS, Cell partition AWS Well-Architected, September 2023. Checked 2026-08-29.
AWS, Cell deployment AWS Well-Architected, September 2023. Checked 2026-08-29.
Kubernetes, Topology Aware Routing Kubernetes documentation. Checked 2026-08-29.
AWS, Overview of Data Transfer Costs for Common Architectures AWS Architecture Blog, 30 June 2021. Checked 2026-08-29.