Storage Tiering

Storage classes trade retrieval latency and cost against storage price. Hot storage is expensive to keep and free to read; archival storage is very cheap to keep and expensive — sometimes slow — to read. Tiering assigns each object to the class matching its likely access, usually driven by age.

It works because access distributions in media and content systems are steeply skewed and largely predictable: heavy access on creation day, occasional access within the week, essentially none after a month, with a small evergreen tail.

Why it matters

Storage is one of the few costs that only ever grows, and it grows with the cumulative history of the product rather than with current activity. Without tiering, a platform pays hot-storage prices forever for data that will never be read again — a cost that compounds annually and eventually dominates the infrastructure bill.

Implementation patterns

• Declarative lifecycle policies rather than application logic, so tiering stays correct as the platform changes.
• Intelligent tiering for the unpredictable tail, paying a small monitoring fee to let the provider move objects on observed access rather than on a policy that cannot know which asset becomes evergreen.
• A CDN in front of everything user-facing. This is what makes aggressive tiering safe: the cold tier is only reached on a genuine cold miss, so its latency is invisible for anything recently or repeatedly accessed.
• Content-addressed keys, which give deduplication for free — valuable on platforms where users copy templates and the same asset exists in enormous numbers of documents.
• Derived artefacts treated as a cache. Thumbnails, previews and rendered exports are regenerable; give them shorter lifecycles and be willing to delete and regenerate rather than paying to retain them.
• Metadata in a database, bytes in object storage — the database holds ownership, permissions, versions and the storage key, and never the content.

Industry example

A design platform storing billions of user assets and rendered outputs faces this in its sharpest form. A single storage class is either ruinously expensive (everything hot) or unusable (everything cold, so the document a user is actively editing takes seconds to load).

The architecture that works is object storage with content-addressed keys and lifecycle policies moving objects through standard, infrequent-access and archival tiers by age — with a CDN absorbing the recently-and-repeatedly accessed traffic so tier latency never reaches a user, and reference counting so deduplicated blocks are only collected when no user references them.

The same structure appears in video platforms tiering by popularity rather than age alone, in observability platforms downsampling and expiring telemetry, and in document platforms archiving inactive workspaces.

Failure scenarios

• Lifecycle thrash. Cold tiers bill a minimum retention period; moving an object down and back up quickly costs more than never moving it. This is a real and frequently-missed cost bug.
• Retrieval latency violating a product expectation. A policy that archives after 30 days is wrong if users routinely reopen documents at 45 days and archival retrieval takes hours.
• Request charges exceeding storage charges. At billions of small objects, per-request pricing can dominate — sometimes making it correct to batch small assets into larger objects.
• Egress ignored. Serving directly from object storage to users is expensive; the CDN is a cost control as much as a latency control.
• Deletion breaking deduplication, where removing one user's asset destroys another user's identical file.

Trade-offs

Tiering buys large, ongoing cost reduction and costs retrieval latency in the tail, retrieval charges, and a policy that must stay aligned with actual product behaviour. It also adds a class of incident that is hard to diagnose — a user reporting slowness for one old document while everything else is fast.

For small datasets the saving does not justify the complexity. For anything where storage is a top-three line item and the access curve decays, it is one of the highest-return architectural decisions available.

Interview question

"Your storage bill has grown 60% year on year while active users grew 20%. Walk me through how you would diagnose it and what you would change — and tell me what could go wrong with the fix."