Origin Shield

Edge caches are numerous by design, and each maintains its own cache independently. When a new object appears, every edge location that wants it misses simultaneously, and the origin receives one request per edge — potentially thousands for a single object.

An origin shield is a small number of caching nodes that all edges fetch through. The shield collapses those concurrent misses: the first request populates it, and every other request waits for and shares that single fetch. The origin sees one request regardless of how many edges wanted the object.

Why it matters

The shield is the difference between an origin sized for the number of distinct objects and one sized for the number of objects multiplied by the number of edge locations and providers. For a static catalogue this is an optimisation. For live streaming it is structural.

In live, every segment is new, produced a few seconds before it is requested, and every edge in the world wants it at the same instant. Without a shield, each new segment generates a synchronised burst of origin requests proportional to the size of the delivery network — and if multiple CDN providers are used, that multiplies again by the number of providers.

Multi-CDN without an origin shield multiplies origin load by the number of CDNs, which is why the two are almost always deployed together and why adding a second CDN "for resilience" without a shield can reduce resilience.

Implementation patterns

• Request collapsing (single-flight) at the shield: concurrent misses for the same key produce one upstream fetch, with the others waiting on it. This is the core mechanism, not the caching.
• A small, deliberately chosen number of shield locations, positioned near the origin, since too many shields reduces the collapse ratio and defeats the purpose.
• Consistent hashing from edge to shield by object key, so all requests for one object converge on the same shield node.
• Shield-level request collapsing for cache revalidation too, not only for misses, since a synchronised expiry produces the same burst.
• Provider-neutral shielding for multi-CDN: a shield the customer operates, in front of the origin, so that all providers collapse through it rather than each having its own.
• Sensible failure behaviour — if the shield is unavailable, edges should fall back to origin directly with rate limiting, since a shield that is a hard dependency has become a single point of failure in front of a highly available origin.
• Capacity planning for the shield tier, which now carries the aggregate miss traffic of the entire network.

Industry example

Origin shielding is standard practice in large-scale live streaming, where record concurrency events — cricket and football broadcasts reaching tens of millions of simultaneous viewers — make the arithmetic unavoidable. A new segment every few seconds, requested by every edge in a global network across multiple CDN providers, produces an origin request rate that no packaging and encoding tier could serve without collapsing.

The pattern also appears in software distribution, container registries and package mirrors, where a popular release produces exactly the same synchronised-miss pattern.

Failure scenarios

• No request collapsing, so the shield caches but does not deduplicate, and the origin still sees a burst for every new object.
• Too many shield nodes, diluting the collapse ratio.
• Inconsistent edge-to-shield mapping, so the same object is fetched by several shield nodes.
• The shield as a hard dependency with no direct-to-origin fallback, adding a failure domain in front of a reliable origin.
• Shield capacity undersized for aggregate miss traffic, making it the bottleneck.
• Synchronised TTL expiry across the network, producing a revalidation storm the shield does not collapse because the requests are technically revalidations rather than misses.
• Multi-CDN added for resilience without a shield, multiplying origin load in the name of reducing risk.

Trade-offs

The shield adds a hop and therefore latency to every cache miss — usually acceptable, since misses are the minority and the alternative is origin overload, but it is a real cost for latency-sensitive small objects.

It adds a tier to operate, monitor and scale, and it concentrates traffic: the shield sees the aggregate miss load of the whole network, so its failure is far more consequential than any single edge's. That requires the shield itself to be redundant, which somewhat undoes the simplicity.

The trade is one extra hop and one more tier in exchange for making origin load proportional to distinct objects rather than to network size. For live streaming and multi-CDN it is not optional. For a small, single-CDN, static workload it is unnecessary machinery.

Interview question

"We are adding a second CDN provider for resilience. Explain what happens to our origin's request rate the moment we do, then design the mitigation — and tell me what happens if your mitigation is itself unavailable."