A content platform's storage costs are dominated by content that is rarely accessed. How should tiering, lifecycle and access patterns be designed?

The shape of the problem

Content popularity follows an extreme curve: a small fraction generates almost all access, and the long tail is requested rarely or never. Storage cost is dominated by the tail, and access cost is dominated by the head, which means the two should be optimised separately.

The tiering design

• Recent and popular content on standard storage, where retrieval is immediate.
• The tail on infrequent-access or archival tiers, accepting a retrieval latency and a per-retrieval charge for content nobody is watching live.
• Lifecycle rules based on access recency rather than on age, since a two-year-old item that is still read weekly belongs in the hot tier and a three-month-old item nobody opened does not.
• Automatic promotion on access, so an item that becomes popular moves back rather than serving repeatedly from cold storage at a per-retrieval charge.

The trap in retrieval pricing

Archival tiers charge for retrieval, and a badly designed access pattern can cost more than the storage saved. A batch job scanning all content monthly, an analytics pipeline reading everything, or a search reindex touching every object will each convert a saving into a loss.

Anything that scans everything must read from a separate copy or from metadata, not from the archived objects — which is a design constraint that must be established before the tiering, not discovered after it.

The derived-data decision

For media, the largest saving is frequently not tiering the source but not generating the derivatives. Producing a full rendition ladder for every upload spends most of the transcoding and storage budget on content nobody watches.

Generate one baseline rendition eagerly and higher renditions on first demand, caching them afterwards. That trades a slower first view of unpopular content — by definition rare — for a large reduction in both transcoding compute and stored bytes.

The refinement that removes the product cost: treat the uploader's own first view as eager, since the creator checking their upload is the worst possible person to give a degraded experience.

The metric

Cost per thousand views and cost per stored item, tracked separately. They move for different reasons and a combined figure hides which of the two interventions is working.