A read-heavy content platform must scale. In what order should caching, replicas, denormalisation, CDN and query optimisation be applied, and what determines the order?
Show the full answer Hide the answer
The order, and the reasoning
1. CDN for anything static or public. For a content platform this is the largest single win, because a large share of traffic is content that does not vary by user. It removes the request from your infrastructure entirely rather than making it cheaper, which is a different order of improvement.
2. Query optimisation. A day of work that frequently removes most of the load: a missing index, an unbounded query, an ORM producing N+1, a large text column pulled into every list view.
3. Application caching of the hot set. Content popularity is extremely skewed, so a small cache gives a large hit rate — which is not true of a uniform workload and is why caching is disproportionately effective here.
4. Read replicas for the long tail, once caching handles the hot set. This buys transparent read scale at the cost of replication lag, which must then be handled explicitly for read-your-own-writes — an author viewing the chapter they just published.
5. Denormalisation and precomputed views for the expensive assemblies: feeds, recommendation lists, author pages. Moves work from read time to write time, which is correct when reads vastly outnumber writes.
What determines the order
Cost per unit of improvement, and reversibility. Each step is roughly an order of magnitude cheaper than the next, and the early ones are trivially reversible while denormalisation is not. Doing them out of order means building expensive machinery to solve a problem an index would have removed.
The skew-specific problem that appears at every layer
Hot keys. One extremely popular item concentrates load on one cache node and one database partition, and adding capacity does not help because the bottleneck is one key on one machine.
The fixes are layer-specific: hot-key replication in the cache, in-process caching for the very top of the distribution, request coalescing so a miss produces one origin fetch rather than thousands, and — on the write side — aggregating view counters in memory and flushing periodically rather than incrementing a single row.
Design for skew from the start. The uniform-distribution assumption is standard in textbooks and false in every content, social and marketplace product, and the cost of being wrong about it is an outage during your most successful moment.