Feature Store  ·  View 13 of 21  ·  Runtime

Materialisation — Four Paths, One Definition

What changes between a batch feature, a streaming feature, an on-demand feature and a backfill — and what does not.

Editable source SVG draw.io All views
Define Compute Write offline Write online Serve Batch features Definition + cadence EMR Serverless Iceberg append Materialise down Latest value Streaming features Definition + window Managed Flink Iceberg append Direct write p99 ≤ 5 s On-demand features Same artefact Replayed at join In path, ≤ 2 ms Backfill Bounded window Isolated pool Rewrite partitions Only if read online No SLO impact Materialisation — One Definition, Four Paths The two blank cells are the design: an on-demand feature has no precomputed value, so parity depends entirely on sharing the artefact. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • Batch features are materialised down from the offline store; streaming features are written to both stores from one computation. Provenance where it is affordable, freshness where it is required.
  • The two blank cells are the architecture: an on-demand feature has no precomputed value anywhere, so its parity depends entirely on sharing the compiled artefact with the training path.
  • Backfill writes online only for features a consumer actually reads online, which is the platform's main cost lever applied at its largest write.

Assumptions

  • Windows from 1 minute to 24 hours, tumbling in the MVP and sliding in phase 2.
  • Backfill of one feature over 13 months ≤ 6 h in an isolated pool, pre-emptible by live jobs.

Risks

  • Dual-write for streaming features has two sinks that can partially fail. The mitigation is the version stamp plus nightly skew replay, not a distributed transaction.
◀  Critical Flow — An ETA Vector Read All views Point-in-Time Training Set Assembly  ▶
Solution Architecture v1.0 · Amazon Web Services · Data Platform Architecture · 2026-09 · 21 views · 14 architecture decision records13-materialisation-by-modality