A social platform serves large volumes of user-generated media with an extremely skewed popularity distribution and a cost-sensitive business. How should the delivery path be designed?

What the skew implies

A small fraction of content generates the overwhelming majority of views, and the rest is requested rarely or never. That has two consequences that pull in opposite directions:

• The hot set is small, so a modest cache gives a very high hit rate. This is the most favourable possible caching profile.
• The long tail is enormous, so storage cost is dominated by content nobody watches, and cache misses on the tail are expensive because they reach origin and cold storage.

The design

• Tier storage by access recency. Recently uploaded and recently accessed content on standard storage; the tail on cold storage with the retrieval latency accepted, because a first view of a two-year-old post can afford a second of extra latency.
• Transcode lazily for the tail. Generating a full rendition ladder for every upload is wasteful when most uploads are never viewed. Generate one baseline rendition on upload, and produce higher renditions on first demand — trading a slower first view of unpopular content for a large reduction in transcoding and storage.
• Aggressive edge caching with immutable URLs, so popular content is served entirely from the edge.
• Origin shielding, so a viral item's global miss is one origin request rather than one per edge location.
• Client-side format and resolution negotiation, so a device receives what it can actually display — serving a higher rendition than the screen can show is pure waste at the scale where it matters most.

The moderation interaction

User-generated media requires a takedown path, and takedown must reach every cache tier, which conflicts with the long TTLs that make delivery cheap. The resolution is a purge mechanism separate from ordinary invalidation — a fast, fail-closed path for legal and safety removals — accepting that it is more expensive and rarely used.

Without it, the choices are short TTLs everywhere (expensive) or content remaining available after removal (unacceptable), and neither is necessary.

The metric

Cost per thousand views, tracked over time, rather than total delivery cost. Total cost rises with growth and hides whether the architecture is improving; the unit metric is the only one that shows whether the lazy transcoding, the tiering and the cache work paid off.