Feature Store  ·  View 17 of 21  ·  Operations

Observability

Five signal types across five stages, and one cell that pays for the platform.

Editable source SVG draw.io All views
Ingest Materialise Store Serve Consume Metrics Stream lag Run duration Partition skew p99 read latency Vector error rate Logs Quarantine events Run audit Rebuild log Reason codes Sampled vectors Traces Run → partition SDK → DynamoDB Model → feature Data quality Null rate, range Freshness SLO Completeness Staleness flag Skew replay verdict Cost Shard hours DPU hours by group GB-month by group $ per M reads Online, zero reads Observability — Signal by Stage The bottom-right cell is the one that pays for the platform: a feature materialised online that nobody has read in 30 days. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • Data quality is a first-class signal row alongside metrics, logs and traces, because this platform's characteristic failure is a plausible wrong number rather than an error.
  • Streaming lag is an SLO, not a dashboard metric: past its threshold a group is marked stale rather than allowed to look fresh.
  • Cost is measured per feature group, which is what makes the online/offline materialisation flag an accountable decision instead of a default.

Assumptions

  • Target ≤ $0.40 per million feature-vector reads; features materialised online with zero reads in 30 days are reported to their owner.
  • Skew and drift measurements retained 13 months so a slow regression can be dated.

Risks

  • Three trace cells are deliberately empty. Tracing a write path through a windowed aggregate is expensive and rarely answers the question; the run audit does it more cheaply.