advanced 2 min answer

A video platform receives a viral upload generating millions of playback requests within minutes. How should edge caching, origin shielding, cache warming, request collapsing, transcoding priority and storage access interact?

vimeoloomcdnorigin-shieldrequest-collapsing
Show the full answer Hide the answer

The sequence of pressure

The first playback request finds nothing cached anywhere. Within a minute, hundreds of edge locations each have thousands of users requesting the same segments. Without collapsing, that is hundreds of thousands of simultaneous origin requests for the same object, and the origin — and behind it, object storage — receives a load that has nothing to do with the actual data volume.

The layers, and what each one removes

  • Request collapsing at each edge. A thousand concurrent requests for one segment produce one upstream fetch and the rest wait. This is the single highest-leverage control and it works at every tier independently.
  • Origin shielding. A designated mid-tier cache sits between all edge locations and the origin, so a global miss becomes one origin request rather than one per location. Without it, collapsing at the edge still leaves you with as many origin requests as you have edge locations.
  • Long TTLs on immutable content. Video segments never change, so they should be addressed by immutable URLs with effectively infinite TTLs. Any cache-busting query parameter here is a direct multiplier on origin load, and inconsistent URL construction is the most common silent cause of a poor hit rate.
  • Range request handling that caches by range consistently, or a client requesting bytes 0-999 and another requesting 0-1023 produce two cache entries for overlapping data.

Transcoding priority

The viral item is often still transcoding when demand arrives. That requires a priority path in the transcoding queue driven by live demand, not just by upload order — and it requires serving the lowest available rendition immediately rather than waiting for the full ladder. A viewer who gets 480p now is better served than one who waits for 1080p.

What the origin must do anyway

Even with everything above, the origin should have a concurrency limit and a shed policy, because the protections are probabilistic and the failure is not. And object storage access should be through the cache hierarchy only — a code path that reads storage directly on a cache miss bypasses every protection you built.

The measurement that tells you it works

Not hit rate alone, which can look fine while the misses are concentrated and fatal. Origin requests per unique object per minute is the number that exposes a collapsing failure, and it should be close to one.