Customer 360 & Real-Time Risk Intelligence Platform  ·  View 18 of 20

Replay and Historical Reprocessing

How a logic change, defect or regulatory restatement is reprocessed from immutable history without disturbing live pipelines.

Editable source SVG draw.io All views
Trigger
Trigger
Business Logic Change
New risk definition
Business Logic Change...
Pipeline Defect
Wrong join or filter
Pipeline Defect...
Regulatory Restatement
Regulatory Restatement
Scope
Scope
Lineage Impact Query
Which tables, which runs
Lineage Impact Query...
Select Time Window
Offsets or partitions
Select Time Window...
Source of truth
Source of truth
Bronze History
Immutable, 10 years
Bronze History...
Kafka Tiered Storage
Offset replay
Kafka Tiered Storage...
Isolated rerun
Isolated rerun
Dedicated Compute Pool
No effect on live SLA
Dedicated Compute Pool...
Shadow Tables
Versioned suffix
Shadow Tables...
New Logic Version
Same code, new tag
New Logic Version...
Compare
Compare
Row & Measure Diff
Old vs new
Row & Measure Diff...
Quality Suite
Full rule set
Quality Suite...
Steward Sign-off
Steward Sign-off
Cut over
Cut over
Atomic Swap
Delta table version
Atomic Swap...
Notify Consumers
Restatement notice
Notify Consumers...
Archive Prior Version
Audit evidence
Archive Prior Version...
unexplained delta
unexplained delta
replay from offset
replay from offset
read partitions
read partitions
approved
approved
Replay and Historical Reprocessing
Replay and Historical Reprocessing
Application we own
Application we own
Risk / gap
Risk / gap
Decision point
Decision point
Data store
Data store
Queue / topic
Queue / topic
Person or role
Person or role
failure / alternate
failure / alternate
event / async
event / async
batch
batch
synchronous
synchronous
Live pipelines keep running throughout; the swap is the only moment consumers observe a change.
Live pipelines keep running throughout; the swap is the only moment consumers observe a change.
v 1.0 · owner Data Engineering · date 2026-08
v 1.0 · owner Data Engineering · date 2026-08Text is not SVG - cannot display

Why this is possible

  • Bronze is immutable and complete, so any downstream state can be regenerated
  • Kafka tiered storage allows replay from an arbitrary offset, not just hot retention
  • Transformations are deterministic functions of input plus a versioned code tag

Isolation

  • Reprocessing runs on a dedicated compute pool with no effect on live SLAs
  • Output lands in versioned shadow tables until it is signed off
  • The atomic swap is the only moment consumers observe a change

Evidence

  • Old-versus-new row and measure diffs are retained as audit evidence
  • Steward sign-off is recorded against the reprocessing run
  • The prior table version is archived, not overwritten
Solution Architecture v1.0 · Enterprise Data Architecture · 2026-0818-replay-reprocessing