Data Quality Service  ·  View 14 of 20  ·  Runtime

Retroactive Evaluation

Re-judging history when a rule is new or a discrepancy is reported, and scoping what has to be disbelieved.

Editable source SVG draw.io All views
Trigger New assertion published or revised Reported discrepancy a human noticed Enumerate Historical versions lookback 30 days Backfill lane below live work Re-judge Evaluate per version time-travel read Verdicts marked retroactive never overwrite Decide Discovery or bad rule? owner must answer FP rate check ≤ 5% / 30 days Scope Corruption window first fail → first pass Lineage walk one hop Mark Downstream suspect persists until cleared Reports + snapshots who read the window Recover Upstream fix producer's job Re-derive downstream then re-judge Window closed explicitly, by a human rule withdrawn clears marks Retroactive Evaluation — Re-Judging History and Scoping the Damage Application we own External / third party Data store Queue / topic Decision point Risk / gap Opportunity failure / alternate event / async A suspect mark is never cleared by the next successful run — only by re-derivation or an explicit human dismissal, because the next run passing says nothing about the window that failed. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • Backfill runs in its own lane below live work. Re-judging history must never delay judging the present, and that is a capacity decision as much as a scheduling one (ADR-10).
  • 'Discovery or bad rule' is an explicit gate with a human on it. A rule that condemns 18 of the last 30 versions is equally likely to be either, and the platform refuses to guess (ADR-10).
  • The corruption window and its blast-radius list are the deliverable. An alert saying 'this table was wrong' without naming who read it is not actionable (ADR-11).

Assumptions

  • 30-day automatic lookback on publication for Tier 1 and 2 assertions. Stated assumption; longer lookbacks are available on request and priced.
  • One-hop suspect propagation in the MVP, with full transitive propagation deferred pending column-level lineage (ADR-11).

Risks

  • Suspect marks accumulate. If nobody clears them, the mark stops meaning anything — which is why clearing is an explicit human act tracked as debt rather than a timeout (ADR-16).
  • Lineage completeness bounds this whole view. A downstream dataset the catalogue does not know about does not get marked, and the blast-radius list will read as complete when it is not.
◀  Advise, Quarantine, Block — and Unknown All views Assertion Lifecycle  ▶
Solution Architecture v1.0 · Databricks Lakehouse on AWS · Data Platform Architecture · 2026-09 · 20 views · 16 architecture decision records14-retroactive-evaluation