Feature Store  ·  View 05 of 21  ·  People and journeys

Journey — Features Go Stale at Peak

19:40 on a Friday, a freshness SLO breaks, and the question is whether the model degrades visibly or silently.

Editable source SVG draw.io All views
ML engineer, on call ETA model, dinner peak Goal — Keep the ETA model honest when a feature stops arriving Trigger — Freshness SLO breach on store_orders_15m at 19:40 on a Friday Done when — The model degrades visibly and recovers, and no wrong ETA reaches a customer as if it were right 1 · Alert 19:40 2 · Triage 3 · Attribute 4 · Mitigate ◆ moment of truth 5 · Recover What they do Take the page Open feature health Check group age Data drift or pipeline? Accept degraded mode Replay from the log Confirm freshness How it feels In control Fine Exposed Where it hurts Drift looks like a deploy Silent zero → wrong ETA What answers it Stale flagged in 60 s Per-consumer health view Version stamp per value Reason code + default 7-day stream log replay Journey — A Model's Features Go Stale at Peak Attribution is the trough: telling a changed world from a changed pipeline is the question the version stamp exists to answer. v 1.0 · owner Data Platform Architecture · date 2026-09

The trough is attribution

  • Detection is cheap and mitigation is policy. The hard phase is telling a changed world from a changed pipeline at nineteen-forty on a Friday.
  • The version stamp written onto every value is what makes that answerable in a minute rather than a morning — it is the difference between 'the data moved' and 'we moved'.
  • Mitigate is the unrecoverable moment: a silent zero here ships a wrong ETA to a real customer, and nothing downstream ever finds out.

Assumptions

  • A group exceeding 2× its freshness SLO is marked stale within 60 s.
  • The stream log retains 7 days, which bounds every replay-based recovery in this journey.

Risks

  • A consumer that declared no criticality gets the platform's default, which may not be the right failure mode for that model.
  • If the stale marker is advisory rather than enforced, a model can read an aged value and never know.
◀  Journey — Ship a Feature All views Layered Architecture  ▶
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-09 · 21 views · 14 architecture decision records05-journey-stale-features