Netflix personalises an entire home page in the time a TV takes to draw a screen. How, given that scoring every title for every member on request is impossible?
Show the full answer Hide the answer
What the interviewer is testing
Whether you reach for the offline/online split — the single most reusable idea in latency-critical personalisation — rather than trying to make the request path faster.
The architecture, as Netflix has published it
Three layers, distinguished by their latency budget:
Offline. Batch computation over full history — model training, candidate generation, similarity computation. Hours available, no latency constraint. Results materialised into stores the request path can read cheaply.
Nearline. Event-driven incremental updates. A member finishes an episode; jobs update the affected precomputed structures within seconds to minutes. This is what gives the system a memory of recent behaviour without a full rebuild.
Online. The request path. Assembles and lightly re-ranks precomputed candidates using immediate context — device, time of day, current session. Strictly bounded latency.
The key move
Match computation to its latency budget rather than trying to speed everything up. The expensive work is not made fast; it is moved to where slowness is free. What remains online is deliberately cheap.
This is CQRS with three freshness tiers, and the shape recurs in search indexing, fraud scoring, pricing, feed ranking and any system where a heavy computation must be served in milliseconds.
The details worth naming
Graceful degradation is designed in. If the online ranker is slow, the page renders from precomputed rows. The worst case is a less relevant home page, never a blank one — personalisation failure is a product degradation, not an outage.
Artwork is personalised too, via contextual bandits — a good example of treating a UI asset as a ranked decision rather than a static file.
Everything is an experiment, which means multiple model versions must be servable simultaneously and every impression attributable to a variant. That requirement shapes the whole architecture and is very hard to retrofit.
What a strong answer adds
Recognising the general design question: for each computation, what is the freshest it needs to be, and what is the cheapest place that can produce it at that freshness? Most systems answer "real time" by default and pay for it everywhere, when the honest answer for most of the computation is "yesterday is fine".