Data Quality Service  ·  View 13 of 20  ·  Runtime

Advise, Quarantine, Block — and Unknown

Five outcome classes, what each does to the data, who is told, and the way back from each.

Editable source SVG draw.io All views
Violation found Attribution What happens to the data Who is told Way back Advise Low severity Rule in probation Dataset or rows Circulates unchanged Scorecard only Promote to block Quarantine Row predicate fails Orphan FK Row-attributable Rows diverted Remainder promoted Owner ticket Re-admit via ingestion Quarantine over ceiling > 2% of the batch Row-attributable Escalates to block Page the owner Fix upstream Block Aggregate fails Reconciliation gap Dataset-level only Version not promoted Last known good served Page + orchestrator skip Override, expiring Unknown Evaluation failed No assertion exists Nothing to attribute Tier 1: not promoted Tier 2-3: circulates Coverage gap on scorecard Author an assertion Three Verdict Actions, and What Each One Does to the Data The bottom lane is the failure class the whole design is aimed at: unknown is rendered as its own state, never folded into pass, and on Tier 1 it withholds promotion exactly as a block does. v 1.0 · owner Data Platform Architecture · date 2026-09

Decisions

  • Quarantine is available only where a violation is row-attributable. Aggregate and reconciliation failures can only advise or block — a fact about the assertion type, not a policy choice (ADR-15).
  • The 2% ceiling escalates quarantine to block, because quarantining a quarter of a batch disguises a broken upstream as partial success.
  • The action is a property of the assertion and tier fixed at authoring time, not a runtime decision: a runtime decision cannot be reviewed before it takes a pipeline down (ADR-15).

The bottom lane is the point

  • 'Unknown' is the failure class the whole design is aimed at. It is rendered as its own state, never folded into pass, and on Tier 1 it withholds promotion exactly as a block does (ADR-05).
  • Its 'way back' is the only one that is not an operational action: someone has to author an assertion, which is why zero-coverage datasets appear on the domain scorecard.

Assumptions

  • 2% quarantine ceiling, 90-day quarantine retention, 7-day default override expiry. Stated assumptions.
  • Re-admission of corrected rows goes through the normal ingestion path; nothing writes into a target dataset directly.
◀  Critical Flow — Commit to Gate All views Retroactive Evaluation  ▶
Solution Architecture v1.0 · Databricks Lakehouse on AWS · Data Platform Architecture · 2026-09 · 20 views · 16 architecture decision records13-advise-quarantine-block