Data Quality Service  ·  View 05 of 20  ·  People and journeys

Journey — Turning a Rule On

A postmortem action item becomes an enforcing assertion, through pricing, backfill and thirty days of probation.

Editable source SVG draw.io All views
Data engineer payments domain Goal — Stop the paise/rupee class of defect reaching the feature store Trigger — A postmortem action item with their name on it Done when — The rule blocks in production and has not paged anyone falsely in 30 days 1 · Draft in Git 2 · Price 3 · Backfill ◆ moment of truth 4 · Probation 30 days 5 · Enforce ◆ moment of truth What they do Writes the assertion Picks severity and tier Reads the dry-run cost Reviews 30 days of results Watches false-positive rate Promotes to block What the platform does Compiles to a bounded plan Refuses block on Tier 3 Estimates $/run Auto-backfills lookback Runs as advise only Tracks FP rate Requires owner approval How it feels In control Unsure Blocked Where it hurts Cost is a guess until it runs 12 of 30 days failed Nobody watches advise verdicts Journey — Turning a Rule On Without Taking a Pipeline Down Probation is the answer to 'a bad rule must not take a pipeline down'; its own weakness — that advise verdicts go unread — is why the false-positive rate is a tracked number rather than a feeling. v 1.0 · owner Data Platform Architecture · date 2026-09

What the platform owes the author

  • A cost before a commitment: the dry-run estimate is posted to the pull request, so an expensive assertion is a conversation rather than an invoice (ADR-08).
  • A verdict on history before enforcement: the would-have-failed report is what turns 'I think this rule is right' into a reviewable claim (ADR-10).
  • Probation as advise-only, then automatic demotion above a 5% false-positive rate — the platform silences a rule that cries wolf, without waiting for a human to notice (ADR-12).

Where it still hurts

  • Phase 4 is the honest weakness: nobody reads advise verdicts, which is why the false-positive rate is a tracked metric with a threshold rather than a dashboard nobody opens.
  • A tier policy refusing 'block' is correct and infuriating. The refusal is drawn because it is where authors argue with the platform.

Assumptions

  • 30-day probation, ≤ 5% false-positive rate over a trailing 30 days as the demotion threshold. Both stated assumptions; the threshold is the single most consequential number in the design.
  • CODEOWNERS-based owner approval is available in the repository hosting the rule DSL.
◀  Journey — The Report That Was Wrong for Three Weeks All views Layered Architecture  ▶
Solution Architecture v1.0 · Databricks Lakehouse on AWS · Data Platform Architecture · 2026-09 · 20 views · 16 architecture decision records05-journey-author-a-rule