Feature Store  ·  View 02 of 21  ·  Context and scope

High-Level Architecture

Six stages from a raw event to a vector a model can trust, with the compiler in the middle rather than at the edge.

Editable source SVG draw.io All views
Sources Event streams Kinesis Warehouse tables Redshift Define Definition repo Git, reviewed Compiler one source, two plans Registry Aurora PostgreSQL Materialise Stream processor Managed Flink Batch engine EMR Serverless Contract guard quarantine on drift Store Offline store S3 + Iceberg Online store DynamoDB, latest only Serve Serving API gRPC on EKS On-demand runtime ≤ 2 ms p99 Training-set service as-of join Prove Serving log 1% sample Skew replay nightly online offline rebuild sample replay Feature Store — High-Level Architecture External / third party Application we own Data store Security / platform batch event / async The compiler is the only place a feature is described. Everything to its right is a materialisation of what it emitted. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • The compiler is the only place a feature is described, and it emits both plans or fails. Everything to its right is a materialisation of what it emitted, not an independent implementation.
  • Serving and materialisation are separate planes with separate capacity. A backfill and a dinner-peak read must never contend.
  • The last stage, Prove, is not monitoring bolted on: replaying sampled serving vectors against the offline path is the only continuous evidence that the two materialisations agree.

Assumptions

  • Streaming features are readable within 5 s p99 of the event; batch groups by 06:00 local p95.
  • 1% of served vectors are logged by default, configurable to 100% per model.

Deliberately omitted

  • The warehouse read path and the online rebuild are drawn in views 09 and 10 rather than crowded in here.
  • Quota enforcement and per-consumer criticality live in the Access layer (view 06), not on this spine.
◀  System Context All views Actors and Journeys  ▶
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-09 · 21 views · 14 architecture decision records02-high-level-architecture