intermediate 2 min answer

The recommendation service is down. Walk through every level of degraded response and say who decides which one is used.

degradationfallbacknetflixproductresilience
Show the full answer Hide the answer

What is being tested

Whether you know the full hierarchy of degraded responses rather than just "fail" and "succeed", and whether you understand that the choice is a product decision.

The hierarchy, best to worst

  1. Fresh personalised recommendations. Normal operation.
  2. Cached recommendations from this user's last session. Slightly stale, still personalised, and usually indistinguishable to the user.
  3. Segment-level defaults — popular in this category, popular for this cohort. Not personal, still relevant.
  4. Editorially curated content. A merchandiser's list. Often performs surprisingly close to personalised, which is worth knowing.
  5. Omit the component entirely. The page renders without the recommendation strip. The product still works.
  6. Show an error in that region, with the rest of the page functioning.
  7. Fail the whole page. Never correct here.

Most systems implement only 1 and 7. All the value is in the middle, and it has to be built deliberately — it does not emerge.

Who decides

Product, in advance, in writing. This is the part engineers most often get wrong by deciding it themselves at 3am.

The question "is a stale recommendation better than none?" is a business question about brand, conversion and user trust. Engineering's job is to state the options and their costs; product's job is to choose, and to have chosen before the incident.

This is precisely what makes Netflix's approach the reference case: when personalisation is unavailable the interface shows popular or curated content rather than an error, and that behaviour was agreed with product and designed, which is why it looks deliberate rather than broken.

Implementation requirements

  • The fallback must not depend on the failing service. Cached recommendations stored in the recommendation service are not a fallback.
  • The fallback data must be maintained. A curated list nobody has updated in two years is a different kind of failure.
  • Degradation must be observable. Emit a metric and alert on it. Silent degradation means serving a stale fallback for four days because nobody noticed — a common and embarrassing outcome.
  • The path must be exercised. A fallback that has never run in production has an even chance of being broken, because nothing tests it. This is a large part of why deliberate failure injection exists.
  • The timeout must be short. Waiting 5 seconds before falling back means the user waited 5 seconds. Fail fast, then degrade.

Where degradation is not acceptable

Naming these is as important as naming the fallbacks:

  • Authorisation — never fail open on a permission check.
  • Payment capture — decline rather than guess.
  • Anything with a legal or safety consequence.
  • Writes that must be durable — accepting a write you cannot persist is worse than rejecting it.

The rule: degrade reads freely, degrade writes cautiously, never degrade correctness for money or access.