Architecture Decision Record

Data Quality Service · Solution Architecture v1.0 · Databricks Lakehouse on AWS · Data Platform Architecture · 2026-09 · 20 views · 16 architecture decision records

The argument these decisions serve is summarised in the Architecture One-Pager.

Sixteen decisions make up this architecture. Everything else across the twenty views is convention, sizing or consequence. Each record states the forcing question, the context that makes it hard, what was decided, how it is realised on Databricks and AWS, the options weighed, what the choice buys and costs, the conditions that would flip it, why it should still be right in ten years, and the lesson worth carrying to a different system.

Status of this document. This is a design, not a report on a running system. The rates, latencies, volumes, retentions and thresholds throughout are stated assumptions, sized for a mid-size consumer super-app's lakehouse — roughly 12,000 datasets, 2.4 PB, 40 TB a day — and chosen to be explicit enough to be argued with rather than vague enough to be unfalsifiable. Where a number is load-bearing, the record says what would change if it were wrong. Nothing here has been measured in production.

How to read a record

• Question: The forcing question: why a decision was needed at all.
• Context: The requirement, the scale and the constraint that make it hard.
• Decision: What this architecture does, stated so it can be checked.
• How it is realised on Databricks and AWS: The concrete mechanism: which service or package, configured how, in which project.
• Options weighed: Chosen, rejected, deferred, or right elsewhere, with the reason for each.
• Consequences: What the choice buys and what it costs, both kept visible.
• Choose differently when: The conditions that would flip the decision for your system.
• Why it holds up over time: What keeps the decision right as scale, staff and technology change.
• Lesson: The principle that transfers beyond this platform.

Decision map

Verdicts and versions: What a judgement is attached to, what the service is allowed to do with it, and what it means when there is none.

• ADR-01 · A verdict binds to an immutable data version, never to the pipeline run
• ADR-02 · The service judges and never repairs; blocking is non-promotion
• ADR-05 · Absence of a verdict is never a pass; unknown and no-coverage are first-class states

Enforcement and circulation: Where the gate sits, how the answer is served to four thousand readers a second, and why a lost event is not a pass.

• ADR-03 · The circulation gate is a separate step that reads published state
• ADR-04 · Current state is a derived projection with a published staleness bound
• ADR-09 · The commit event stream is an optimisation; the reconciliation sweep is the guarantee

Evaluation and cost: Whose compute does the judging, how a rule's price is bounded before it runs, and what happens when the budget runs out.

• ADR-06 · An assertion must compile to a bounded, fusable plan or it is refused
• ADR-07 · Computation goes to the data, on declared priority lanes
• ADR-08 · Cost is bounded per dataset, priced at authoring time, and capped in aggregate

History and blast radius: Re-judging the past, and how far the consequences of a discovery are allowed to travel.

• ADR-10 · Retroactive evaluation runs in its own lane, behind a mandatory human judgement
• ADR-11 · Suspect propagates one hop; the blast radius is a report, not a cascade

Credibility and debt: Keeping rules worth believing, keeping authority with the owner, and making every exception end.

• ADR-12 · A bad rule cannot take down a pipeline: probation, circuit breaking and automatic demotion
• ADR-13 · Blocking authority belongs to the dataset owner, and the action is fixed at authoring time
• ADR-16 · Every escape hatch is an expiring object that creates a debt item

Governance and data handling: Why a verdict must not leak what it judged, and why quarantine is custody rather than storage.

• ADR-14 · A verdict must not become a side channel
• ADR-15 · Quarantine only where violations are row-attributable, with a ceiling that escalates to block

Technology by capability

Every capability below names the choice, where it comes from, the alternative that was weighed, and why this one won. The stack is Databricks Lakehouse on AWS because the topic genuinely belongs to the table layer — a verdict bound to an immutable table version needs a format that provides one — and because the practice's recent use cases have leaned on open-source-on-premises and Azure; nothing in the requirement is written to a vendor's service catalogue.

The decisions, and the alternatives that lost

Verdicts and versions

What a judgement is attached to, what the service is allowed to do with it, and what it means when there is none.

ADR-01 · A verdict binds to an immutable data version, never to the pipeline run

Status: Accepted · Shown on views: 10, 11, 14

When the service records that a dataset passed or failed, what exactly is the subject of that claim — the run that produced the data, or the data itself?

Context. Every data quality tool in common use records run outcomes, because that is where the check executes: the pipeline calls the validator, the validator returns pass or fail, and the result is written against the run id. It reads perfectly well and it is almost useless six months later. A run-keyed verdict cannot be re-derived, because the run is gone. It cannot be revised, because a rule written later has nothing to attach to. And it cannot answer the only question that matters during a real incident — 'which weeks of this table should we disbelieve' — because the verdicts describe executions rather than data. Meanwhile the lakehouse already provides exactly the primitive needed: every commit produces an immutable, addressable version of the table, readable long after the job that wrote it has been deleted.

Decision. Every verdict is keyed by the pair (dataset version, assertion version) and is immutable once written. A verdict names the exact data it judged and the exact rule that judged it. Re-evaluating the same pair is idempotent and produces no second row. A revised judgement — a corrected rule, a re-run after an engine upgrade — is a new verdict that supersedes an earlier one, and both remain readable. The run that triggered the evaluation is recorded as metadata on the verdict, never as its key.

How it is realised on Databricks and AWS. Delta Lake table versions are the subject: the evaluator reads a specific version through time travel, and the verdict row carries version_id as a foreign key alongside assertion_ver_id. Verdicts land in an append-only Delta table on S3 partitioned by evaluation date, with the (version, assertion version) pair unique. Unity Catalog supplies the version history the reconciliation sweeper diffs against. The same evaluator code path serves live and retroactive evaluation, which is what makes the two produce identical verdicts — and is the property the four-week prototype is asked to demonstrate.

What it buys

• Any past state of any table can be re-judged, so a rule written today can say which of last month's versions to disbelieve.
• A corruption window is computable as an interval between two versions rather than reconstructed by hand.
• The ML engineer's question — 'was the snapshot I trained on sound' — has an answer years later.
• Re-evaluation is idempotent, so a re-run after an engine upgrade cannot silently double-count or contradict itself.

What it costs

• A verdict store sized by versions × assertions rather than by runs — roughly 180,000 rows a day at the assumed scale.
• A hard dependency on the table format keeping old versions available for the full 25-month retention window.
• A vacuum policy that must never delete a version a retained verdict still references, which is a coupling somebody has to own.

Choose differently when. If the storage layer stopped providing addressable immutable versions — a migration to a format without time travel, or a retention policy that vacuums aggressively enough to make the lookback window meaningless — the decision would have to change, and with it the whole retroactive capability. The honest signal to watch is the gap between the verdict retention window (25 months) and the table history actually retained; if the second is shorter, the first is a promise the platform cannot keep.

Why it holds up over time. This is the decision most likely to still be right in ten years, because it asks almost nothing of the technology: one immutable, addressable version of a dataset, which Delta, Iceberg, Hudi and even a manifest-plus-objects layout all provide. What would break it is not a migration but convenience — a future team joining verdicts to a job id because the pipeline already had one to hand. That is why the version key appears in the data model view rather than in an implementation note.

Lesson. Bind a judgement to the thing being judged, not to the process that produced it. The same mistake shows up in security scanning keyed to build ids rather than image digests, and in test results keyed to CI runs rather than commits: in both cases the record stops being useful at exactly the moment someone needs to ask a question about the past.

ADR-02 · The service judges and never repairs; blocking is non-promotion

Status: Accepted · Shown on views: 01, 09, 13

When the service finds bad data, may it change anything — fix a value, drop a row, roll back a version — or may it only decline to let the data circulate?

Context. The tempting scope creep is repair. The service already knows the rule, has the offending rows in hand, and could coerce the currency, null the impossible timestamp, or roll the table back to the last good version. Every one of those makes the quality service a writer to the estate it judges, which has three consequences that arrive later and hurt more than the original defect: the data no longer matches what the producer sent, so the producer cannot reproduce it; the evidence of what went wrong is destroyed by the fix; and the service becomes a component whose bug corrupts production data rather than one whose bug delays a verdict. Rollback is the subtlest version of the same mistake — it looks like restoring a good state, and it removes the version an auditor would need to see.

Decision. The service has no write path into any judged dataset. A failing dataset-level assertion results in the new version not being promoted to the consumable state; consumers continue reading the last known good version, and the failed version remains in the table history, inspectable. A failing row-level assertion results in offending rows being diverted to a quarantine dataset and the remainder promoted — a write to quarantine, never a modification of the source. Repair happens upstream, through the producer's normal ingestion path, and the corrected data is judged again from scratch.

How it is realised on Databricks and AWS. Promotion is a property change, not a data movement: the consumable pointer for a dataset advances to a version only when its state permits. A blocked version stays in Delta history and is reachable by version number for anyone inspecting it. Quarantine is a separate Delta table under the source dataset's classification, holding the diverted rows plus the verdict context. Re-admission of corrected rows goes back through the ingestion pipeline, so quarantined rows enter the target exactly as any other rows do, and are judged accordingly.

What it buys

• The service's failure modes are 'late verdict' and 'wrong verdict', never 'lost data' — which is what makes it safe to depend on.
• Every rejected version survives and stays inspectable, so an investigation has evidence rather than a description.
• The producer's output is never altered, so a defect stays visible to the team that can fix its cause.

What it costs

• Consumers can be served stale-but-good data for as long as the producer takes to fix the cause, which on a Tier 1 dataset is felt immediately.
• That cost is the whole reason the override mechanism in ADR-16 exists, and therefore the reason quality debt exists at all.
• Re-admission through the ingestion path is slower than writing corrected rows directly.

Choose differently when. If the platform ever genuinely needs automated remediation — a deterministic, reversible, well-understood coercion applied at scale — it should be a separate service with its own identity, its own audit trail, and its own verdicts produced by this one. Folding it into the judge would be the change to resist, not the capability itself.

Why it holds up over time. The separation of judging from repairing is a boundary, not a technique, and boundaries age well. The pressure on it will come from a specific, reasonable request — 'just trim the whitespace, it fails validation every night' — and the right answer will still be that the producer trims the whitespace and the judge records that they did.

Lesson. A component that both detects a problem and fixes it can hide the problem it fixed. Keeping detection and remediation in different systems costs coordination and buys the ability to know what actually happened.

ADR-05 · Absence of a verdict is never a pass; unknown and no-coverage are first-class states

Status: Accepted · Shown on views: 12, 13, 18

What does the platform say about a dataset that has not been evaluated, could not be evaluated, or has no assertions at all?

Context. This is the defect that produces the silent-corruption case, and it is almost always an interface decision rather than an engineering one. A dataset with forty passing assertions and a dataset nobody has ever written a rule for both render as a green tick, because the underlying model has two values and reality has at least five. The same collapse happens in the failure path: an evaluation that timed out, a table whose metadata could not be read, a freshness assertion that has not run for three cadences — each of these is an absence of information, and each is routinely displayed as health. The analyst in view 04 publishes a wrong number for three weeks not because a check failed but because the absence of a check looked identical to a passing one.

Decision. The state model has five values — good, degraded, suspect, blocked, unknown — and coverage is a separate, always-rendered dimension. An evaluation that cannot complete produces degraded; one that cannot start or cannot read its target produces unknown; a dataset with no assertions is reported as zero coverage and never as passing. A Tier 1 dataset in unknown state is not promoted. A freshness assertion that has not been evaluated within twice its cadence is treated as breached rather than as silent.

How it is realised on Databricks and AWS. The state index carries the state and a coverage summary in the same record, so no caller can read one without the other. The scheduler marks a newly committed version unknown before evaluation begins, so the window between commit and verdict is explicitly not-yet-judged rather than implicitly fine. Absence checks in view 18 — no work enqueued, no verdict in two cadences, climbing unknown count, datasets with zero assertions — are the signals that make this decision operational rather than aspirational.

What it buys

• An unmonitored dataset looks unmonitored — the single property that prevents the silent-corruption case.
• Fail-safe behaviour on Tier 1 becomes expressible, because 'we have not judged this' is a state the gate can act on.
• Absence checks have something to assert against: a climbing unknown count is a signal rather than a gap.

What it costs

• Every consumer has to handle five states rather than two, which needs a client library that makes the safe default easy.
• Real friction on Tier 1: a dataset whose evaluation plane had a bad hour is not promoted, and someone is paged about a delay rather than a defect.
• The console has no two-valued summary to fall back on, which will be argued about by whoever wants a green badge.

Choose differently when. Nothing short of a different problem would flip this. The decision to keep watching is whether five states is too many in practice: if consumers collapse them into two in their own code, the platform has the model right and the client library wrong.

Why it holds up over time. Of every decision here, this is the one most independent of technology and most likely to be quietly eroded — by a well-meaning change that makes the console's summary badge green when there is nothing to report. The defence is that the state model itself has no two-valued projection to fall back on.

Lesson. Distinguish 'we know it is fine' from 'we do not know'. Any monitoring system that renders those the same will eventually report health for something it stopped watching, and the gap is discovered by a customer rather than by the system.

Enforcement and circulation

Where the gate sits, how the answer is served to four thousand readers a second, and why a lost event is not a pass.

ADR-03 · The circulation gate is a separate step that reads published state

Status: Accepted · Shown on views: 02, 12, 18

Does the quality service itself promote table versions — owning the final step of every write so nothing can circulate unjudged — or does it publish a state that each pipeline and consumer is expected to consult?

Context. This is the sharpest trade-off in the design, and both answers are defensible. If the service owns promotion, enforcement is airtight: there is no path to consumable data that does not pass through the judge. It also makes the quality service a hard availability dependency of every write in the platform, so an outage in a system whose job is to be careful becomes an outage in the systems whose job is to deliver. If the service only publishes state, it stays off every critical path and can be as slow or as unavailable as its own SLO allows — but enforcement becomes a matter of each consumer choosing to ask, and a gate that can be skipped will be skipped during exactly the incident where it matters. The naive framing treats this as a governance question. It is an availability question.

Decision. The quality service publishes claims about versions; promotion is a distinct step, owned by the platform's write path, that consults the published state before advancing the consumable pointer. The service does not perform promotion. To keep this from decaying into advisory enforcement, three things are structural: promotion is implemented once, in the shared ingestion framework, rather than by each pipeline; a Tier 1 dataset in unknown or blocked state cannot be promoted by that framework; and gate reads are themselves instrumented, so a consumer that used to ask and has stopped asking raises an alert.

How it is realised on Databricks and AWS. The shared ingestion framework calls the State API before advancing a dataset's consumable pointer, and the orchestrator calls it before starting any dependent job. The state index is a DynamoDB global table read directly by both, so the read path does not traverse the control plane. The 'no gate reads from a consumer' absence check in view 18 is what makes the arrangement observable rather than assumed.

What it buys

• A quality-service outage means 'no new verdicts', not 'no writes', so the control plane can be deployed and restarted freely.
• The state index is readable independently of the control plane, so consumers keep getting answers during an outage.
• Promotion implemented once in the shared framework means the gate's behaviour is one piece of code rather than forty.

What it costs

• Enforcement depends on the ingestion framework being the only promotion path, which is an organisational property as much as a technical one.
• The day a team publishes a consumable table outside the framework, that dataset is ungated and the platform will not know unless coverage is measured against the catalogue.
• 'No gate reads from a consumer that used to read' has to be an alert, because the weakness is silent by construction.

Choose differently when. Two signals would flip it: a repeated incident where data circulated because a promotion path bypassed the framework, or a regulatory requirement that makes 'we published the state and they did not read it' an unacceptable answer. Either would justify moving promotion inside the service and accepting the availability coupling — which is why the state index is already readable independently of the control plane.

Why it holds up over time. The boundary is the durable part; where the gate physically sits is not. Stating them separately is deliberate, so that a future team can move the gate without accidentally also merging the judging and circulating planes — the change that would make a quality outage a platform outage.

Lesson. Enforcement and availability are usually the same trade wearing two different vocabularies. A control that sits on the critical path is stronger and more expensive in exactly the same proportion, and choosing consciously beats discovering it during an incident.

ADR-04 · Current state is a derived projection with a published staleness bound

Status: Accepted · Shown on views: 08, 09, 16

Where does the answer to 'is this dataset safe to read right now' live, and is a consumer allowed to act on an answer that is slightly out of date?

Context. The read/write ratio decides this. Verdicts are written a couple of times a second at the assumed scale; state is read four thousand times a second, mostly by orchestrator tasks asking about inputs before they start. Serving those reads from the verdict store means an analytical store on a latency-sensitive hot path, which it will do badly. Serving them from a strongly consistent transactional store means the platform's highest-volume read path shares a failure domain with the control plane. The uncomfortable part is that any projection can be stale, and during the seconds after a block is issued a consumer can read 'good' for a version that has just been condemned. Pretending otherwise is the actual risk; consumers will build on whatever consistency they observe rather than whatever the documentation claims.

Decision. The current-state index is a derived projection of the verdict store, keyed by dataset and partition, holding the state and the last-known-good version pointer. Every response carries an as_of timestamp and therefore an explicit staleness, and the API publishes a bound on it. Consumers are required to treat a stale answer past the bound as unknown rather than as the last value seen. The index is never written to directly: it is only ever derived, which is what makes rebuilding it a routine operation rather than a recovery event.

How it is realised on Databricks and AWS. Verdicts land in Delta; a streaming derivation writes the state index into a DynamoDB global table, replicated to a second region with a target lag of two seconds. The State API reads DynamoDB directly and does not traverse the control plane, so a control-plane outage leaves consumers able to get an answer. The index's recovery objective is deliberately asymmetric — RPO 15 minutes, RTO 10 minutes — because its recovery is a rebuild from verdicts rather than a restore from a backup.

What it buys

• A read path that absorbs 12,000 reads a second at peak without touching an analytical store.
• Regional redundancy for the one question consumers must always be able to ask.
• Recovery by rebuild rather than restore, which is why the index carries a 10-minute RTO against a 15-minute RPO.

What it costs

• An eventual-consistency window in which a consumer can read 'good' for a version that has just been blocked.
• That window has to be honest: a published staleness bound, a stale-index alert at 60 seconds, and consumers required to degrade to unknown past the bound.
• Two stores to reason about, and a derivation whose lag is a first-class metric rather than an implementation detail.

Choose differently when. If a class of consumer appeared for which any staleness is unacceptable — a real-time financial control reading a Tier 1 aggregate — the answer is a strongly consistent read path for that narrow case, not the removal of the projection for everyone. If the catalogue gains a property API that can absorb the write volume, option four becomes the better answer because it puts the state where consumers already look.

Why it holds up over time. Separating the durable record from the serving projection is one of the few patterns that has survived every generation of data infrastructure, and the reason is the same each time: the two have different read shapes, different retention needs and different recovery objectives. What will change is which technology holds the projection.

Lesson. Publish the staleness alongside the answer. A consumer that knows an answer is eight seconds old can decide what to do; a consumer told the answer is authoritative will build as though it is, and the platform inherits a bug it cannot see.

ADR-09 · The commit event stream is an optimisation; the reconciliation sweep is the guarantee

Status: Accepted · Shown on views: 02, 07, 18

If the platform learns about new table versions from an event stream, what happens to a version whose event was never delivered?

Context. Event-driven evaluation is the obvious design and the right one for latency: a commit notice arrives, work is enqueued, a verdict appears in ninety seconds. The failure mode is quiet. An event lost to a consumer group rebalance, a topic retention edge, a producer that wrote without emitting, a bug in the notice path — any of these leaves a version that was never judged, and under ADR-05 that version is unknown rather than passing, which is safe but only if something eventually notices. Relying on the stream alone means the platform's correctness depends on a delivery guarantee it does not control, and the gap is invisible precisely because nothing failed.

Decision. Evaluation has three independent triggers and the event stream is the fastest, not the authoritative one. A reconciliation sweeper compares the catalogue's table version history against the set of evaluated versions on a fixed ten-minute cadence and enqueues anything unjudged. Scheduled evaluation runs regardless, at each tier's cadence. The sweeper's catch rate is a published metric: a rising catch rate means the event path is degrading, which is information the event path itself cannot supply.

How it is realised on Databricks and AWS. The sweeper queries Unity Catalog for versions committed since its last watermark per dataset, left-joins against the verdict store's distinct version ids, and enqueues the difference at normal priority. It is deliberately cheap — a metadata query, not a scan — so it can run every ten minutes across 12,000 datasets. Its own failure is caught by an absence check: no work enqueued by the sweeper for two cycles is itself an alert.

What it buys

• A correctness property that does not depend on any broker's delivery guarantee.
• Sweeper catch rate as a metric that makes degradation of the fast path visible — something the fast path cannot report about itself.
• A bounded worst case: ten minutes from an undelivered commit notice to an enqueued evaluation.

What it costs

• A second trigger path that must stay behaviourally identical to the first, or the two produce different verdicts.
• A metadata query every ten minutes across 12,000 datasets, cheap but not free.
• A ten-minute window in which a missed version sits unjudged, tolerable only because 'unknown' already withholds promotion.

Choose differently when. Nothing about a better message broker would flip this; the decision is about not depending on any broker's guarantee. What would change is the cadence, which is a cost-versus-latency dial rather than an architectural choice.

Why it holds up over time. Reconciliation against the source of truth is the pattern that has outlived every generation of messaging, and for the same reason each time: a system that cannot detect its own missed work will eventually have missed work it cannot detect. The sweeper is cheap insurance against a class of bug nobody can enumerate in advance.

Lesson. If a pipeline's correctness depends on receiving every event, add the thing that notices a missing one. The reconciliation loop is usually a fraction of the work of the fast path and it is the only part that makes the fast path's failures visible.

Evaluation and cost

Whose compute does the judging, how a rule's price is bounded before it runs, and what happens when the budget runs out.

ADR-06 · An assertion must compile to a bounded, fusable plan or it is refused

Status: Accepted · Shown on views: 06, 12, 17

Is an assertion arbitrary code the platform runs and hopes finishes, or a declaration the platform must be able to cost and combine before accepting it?

Context. The expressive answer is to let authors write code: a Python function, a Spark job, whatever catches the defect. It maximises what can be expressed and it makes the platform's cost and latency unpredictable per assertion, which at 40,000 assertions means unpredictable in aggregate. It also makes fusion impossible — forty assertions on one table become forty scans, and the cost model collapses. The subtler problem is diagnosis: when an arbitrary function fails, the platform cannot tell whether the data is bad or the function is, and the resulting verdict is unattributable. Meanwhile the vast majority of real assertions — schema shape, row predicates, aggregate bounds, null ratios, referential integrity, distribution comparison — are expressible declaratively and fuse into a single pass.

Decision. Assertions are authored in a declarative DSL and compiled to engine-native query plans. The compiler rejects any assertion it cannot bound: no correlated subquery over an unpartitioned table, no cross join without a declared key, no user-defined function whose cost cannot be estimated. All assertions targeting the same dataset and partition are fused into a single scan wherever their plans permit it, and the achieved fusion ratio is reported per dataset. Metadata-only assertions — schema, freshness — are evaluated without reading data files, and a schema failure short-circuits the remaining assertions for that dataset, because row predicates against a changed schema produce meaningless verdicts.

How it is realised on Databricks and AWS. The DSL compiles to Spark SQL executed on a Databricks SQL warehouse. Compilation produces a plan and an estimated scan volume; the estimate is posted to the pull request (ADR-08) and the plan is cached. Fusion is a rewrite that turns N predicates into N aggregate expressions over one pass, with row-level violations collected as identifiers rather than as rows. Escape-hatch assertions that genuinely need custom logic are supported as a declared, separately budgeted assertion type, priced at full-scan cost and refused on Tier 1 unless explicitly approved.

What it buys

• A cost model that still holds at 40,000 assertions, because cost grows with data scanned rather than with rule count.
• One scan per partition instead of forty, with the achieved fusion ratio reported so the saving is visible.
• Verdicts attributable to the data rather than to check code, because the platform knows what each assertion asked.

What it costs

• A DSL to design, document and evolve — a real and permanent maintenance burden.
• A class of author who will find the restriction arbitrary until someone else's unbounded assertion would have consumed their budget.
• An escape hatch for genuinely novel checks, which has to be priced at full-scan cost and watched as a share of the whole.

Choose differently when. If the engine gained genuinely reliable cost estimation for arbitrary functions, the restriction would be worth loosening. The signal to watch is the escape-hatch share: if more than a small minority of assertions need it, the DSL's vocabulary is wrong and the fix is to extend the DSL rather than to widen the hatch.

Why it holds up over time. The principle — declare what you want checked, let the platform decide how — long outlives any particular DSL, and it is the same reason SQL outlived the query languages that let you specify the join order. What will date is the vocabulary, which is why assertions are versioned objects that can be recompiled.

Lesson. A platform can only optimise what it can understand. Accepting opaque work units buys expressiveness and permanently forfeits batching, costing and attribution — a trade worth making explicitly rather than by default.

ADR-07 · Computation goes to the data, on declared priority lanes

Status: Accepted · Shown on views: 07, 16, 18

Does evaluation run in the same engine the pipelines use, or in compute dedicated to quality work?

Context. Pushing down into the platform's own engine is cheap, avoids moving 2.4 PB anywhere, and inherits the governance context for free — the evaluator sees exactly what the catalogue's grants allow, with no second access model to maintain. Its defect is timing: quality work then competes for capacity with production pipelines, and the competition is worst during an incident, which is precisely when verdicts matter most. Dedicated compute removes the contention and adds a duplicated cost line, its own credential path into every dataset in the estate, and — if data has to move to reach it — a second copy of regulated data with its own classification problem. The choice turns on whether evaluation is elastic-priority work or carries a hard SLA, and the honest answer is that it is both, at different tiers.

Decision. Evaluation is pushed down into the lakehouse engine; no dataset is copied out of its storage layer to be judged. Contention is managed by lanes rather than by isolation: a reserved-capacity live lane for Tier 1 and Tier 2 work, a shared lane for Tier 3, and a separate low-priority lane on interruptible capacity for retroactive backfill. Per-domain concurrency quotas stop a single domain's rule set from starving the rest, and work is partitioned per dataset so one very large table cannot monopolise the lane.

How it is realised on Databricks and AWS. Databricks SQL warehouses run the live lane with reserved capacity and autoscaling; job clusters handle distribution and reconciliation work; the backfill lane runs on spot instances, where interruption is genuinely acceptable because a delayed retroactive verdict has no consumer waiting on it. Unity Catalog grants are enforced during execution rather than re-implemented, and the evaluator's workload identity is scoped to the assertion's registered targets (ADR-14).

What it buys

• No dataset movement at 2.4 PB, and catalogue grants inherited rather than re-implemented.
• A single access model, so there is no second set of credentials into the estate to secure and rotate.
• Contention bounded by reservation per tier, with backfill on capacity where interruption is genuinely acceptable.

What it costs

• Reserved capacity that sits idle between peaks and shows up as a cost line nobody likes.
• A residual risk left in the open: if the Tier 1 lane cannot be held under incident load, verdict latency degrades exactly when it matters.
• That risk is only measurable during a real incident, not in steady state.

Choose differently when. Two things would flip it: a sustained inability to hold the Tier 1 lane under incident load, or a regulatory requirement that quality evaluation be demonstrably independent of the pipelines it judges. Either justifies dedicated compute and the credential-surface cost that comes with it.

Why it holds up over time. 'Computation goes to the data' has been the right default for as long as data has been too big to move, and nothing on the horizon changes the arithmetic. What will change is the lane mechanism — reservations, queues, warehouse pools are all implementation — which is why the decision is stated as lanes by tier rather than as a product feature.

Lesson. Sharing infrastructure is cheap until the moment both workloads need it, and that moment is rarely random — it is usually the incident that made both urgent. Reserve for the tier that cannot wait, and be honest that the reservation is the whole mitigation.

ADR-08 · Cost is bounded per dataset, priced at authoring time, and capped in aggregate

Status: Accepted · Shown on views: 17, 18, 16

What stops quality coverage from becoming the line item that gets cut, and what happens when a dataset's assertions cannot finish inside their budget?

Context. Data quality spend has a predictable arc: coverage grows, cost grows with it, someone notices the bill, and coverage is cut by whoever can cut it fastest — usually by disabling the expensive assertions, which are disproportionately the full-scan reconciliation and distribution checks that catch silent corruption. The cut is invisible because nothing fails. Meanwhile the per-run failure mode is just as damaging: an assertion set that cannot finish inside its window either delays every other dataset behind it or produces nothing, and 'produces nothing' has already been ruled out as a pass by ADR-05.

Decision. Three controls, at three timescales. At authoring time, an assertion's cost is estimated from its compiled plan and posted to the pull request, so an expensive rule is a conversation before it is an invoice. At run time, each dataset has a declared evaluation budget; exceeding it degrades to the metadata-only assertion set — schema and freshness first — and emits a degraded verdict rather than no verdict. In aggregate, total evaluation cost is held at or below 2.5% of the data platform's compute spend, attributed per domain and published alongside each domain's quality score, so that coverage and its price are always read together.

How it is realised on Databricks and AWS. The plan compiler produces an estimated scan volume, converted to an estimate in currency using current warehouse rates and posted as a pull-request comment. Run-time budgets are enforced by the scheduler, which submits the metadata set first and the scan set second, so degradation is a truncation rather than an abort. Cost is attributed by tagging warehouse usage per dataset and rolling it up per domain; the full-scan fraction per dataset is reported so that the cheap path is visibly the default.

What it buys

• Predictable spend, and a price for coverage that is visible at the moment someone reviews the rule.
• Budget exhaustion becomes a graded degradation that keeps schema and freshness, rather than an absence of information.
• Cost published next to the quality score, so coverage and its price are always read together.

What it costs

• Authoring friction, and a cost model that must be maintained as warehouse pricing changes.
• The honest possibility that a dataset's full assertion set does not fit its budget, which surfaces as a coverage decision somebody has to make.
• A 2.5% ceiling that becomes unachievable with full coverage if the fusion ratio proves lower than assumed.

Choose differently when. If the fusion ratio proves low in practice — the prototype's second test — the 2.5% ceiling stops being achievable with full coverage, and the choice becomes explicit: raise the ceiling or reduce Tier 1 coverage. That is a decision for the platform's owners, and the numbers here exist so it is made with numbers rather than with a feeling.

Why it holds up over time. Pricing a control alongside the risk it mitigates is the only way a control survives a cost review, and that will be true in any decade. The specific ceiling is an assumption; the practice of publishing cost next to coverage is the durable part.

Lesson. Any safeguard whose cost is invisible and whose benefit is invisible will eventually be cut. Publishing both together is what makes the trade someone's decision rather than someone's oversight.

History and blast radius

Re-judging the past, and how far the consequences of a discovery are allowed to travel.

ADR-10 · Retroactive evaluation runs in its own lane, behind a mandatory human judgement

Status: Accepted · Shown on views: 14, 17, 15

When a new or revised assertion is published, does the platform evaluate it against history — and if it fails eighteen of the last thirty versions, who decides whether that is a discovery or a bad rule?

Context. ADR-01 makes retroactive evaluation possible; this decision is about whether it is automatic and what happens to the result. Not backfilling means every rule starts from today and the corruption that has been running for a month stays invisible until someone thinks to ask. Backfilling automatically and then enforcing on the result means a mistyped rule can retroactively condemn a month of a Tier 1 table and mark half its downstream, which is a self-inflicted incident. The two cases are genuinely indistinguishable from the platform's side: a rule that fails eighteen of thirty past versions is either the most valuable thing the platform has produced this quarter or a bug in a predicate, and only someone who knows the domain can say which.

Decision. Publishing an assertion automatically backfills it across a declared lookback window — thirty days for Tiers 1 and 2 — and produces a would-have-failed report. The assertion cannot be promoted from advise to an enforcing action until the report has been reviewed and the discovery-or-bad-rule question answered explicitly by the dataset owner. Backfill runs in a separate low-priority lane on interruptible capacity, so re-judging history never delays judging the present. Retroactive verdicts are stored alongside live ones and are clearly marked as retroactive.

How it is realised on Databricks and AWS. Backfill enumerates the dataset's versions inside the lookback window, reads each through Delta time travel, and runs the identical evaluator used on the live path — which is why the four-week prototype is asked to prove the two produce byte-identical verdicts. Results are written as verdicts with retroactive = true and summarised into a report posted back to the pull request that introduced the rule. The lane runs on spot warehouses; an interrupted backfill resumes from its watermark.

What it buys

• The platform's most valuable single capability: a rule written today tells you which of last month's versions to disbelieve.
• A rule's first real feedback is about history, which measurably changes how authors write them.
• Backfill on its own interruptible lane, so the capability costs nothing in live verdict latency.

What it costs

• Compute on every publication, bounded by the lane and the lookback but not zero.
• A mandatory review step before any new assertion may enforce, which authors will feel as a delay.
• A verdict store that grows with backfill volume as well as live volume.

Choose differently when. If the human gate becomes a bottleneck — rules queuing for review for weeks — the right response is a narrower gate (only where the backfill failure rate exceeds a threshold) rather than removing it. If the lookback proves too short to catch the defects that matter, it is a cost dial, not an architectural change.

Why it holds up over time. The capability is durable because it follows from ADR-01 rather than from any tool. The gate is durable for a different reason: no amount of model-assisted rule authoring removes the need for someone who knows the domain to say whether a month of failures is news. If anything, generated assertions make the gate more necessary.

Lesson. When a system can produce two opposite conclusions from the same evidence — we found something, or we broke something — do not let it act on either. Produce the evidence, name the ambiguity, and route it to whoever can resolve it.

ADR-11 · Suspect propagates one hop; the blast radius is a report, not a cascade

Status: Accepted · Shown on views: 14, 10, 04

When a corruption window is found in one dataset, how far does the resulting suspicion travel through the lineage graph?

Context. Correctness argues for full transitivity: if a fact table was wrong for three weeks, everything derived from it during those weeks is also suspect, and everything derived from those, and so on. Applied to a real warehouse, one upstream defect in a payments fact table marks several hundred downstream objects, at which point the mark conveys nothing and consumers learn to ignore it — alert fatigue, one layer down, with the same ending. Stopping at one hop is arbitrary and bounded, and it leaves genuinely affected objects two hops away unmarked. Neither option is right; what makes the choice tractable is that the platform can compute the full transitive set as a report without applying it as state.

Decision. State propagation stops at one hop: datasets directly derived from the affected versions are marked suspect. The full transitive closure is computed and published as part of the corruption-window report, together with the reports and feature-store snapshots that read the affected versions, and the lineage coverage percentage behind the calculation. Marks are never cleared automatically by a subsequent successful run; they are cleared by re-derivation or by an explicit human dismissal, and an uncleared mark ages as a debt item.

How it is realised on Databricks and AWS. The blast-radius resolver walks Unity Catalog lineage from the affected dataset versions. One-hop results are written to the state index as suspect marks with a hop count and a null cleared_ts; the transitive set and the consumer list go into the report only. Median mark age is a published metric, because it is the signal that says whether the propagation depth is right.

What it buys

• A suspect mark that still means something after a year, because it has not been applied to half the warehouse.
• A report that is honest about its own completeness, since lineage coverage is stated alongside the blast radius.
• Median mark age as the signal that says whether the chosen depth is right.

What it costs

• A real two-hop gap: an object two steps downstream is in the report and not in the state, so a consumer reading state alone can miss it.
• Clearing marks is a human workflow, which is the part most likely to be neglected.
• The gap has to be stated on the report rather than hidden, which makes the deliverable less satisfying than it looks.

Choose differently when. Column-level lineage is the flip condition. With it, transitive propagation becomes precise enough to be useful — 'this column in this downstream table is suspect' rather than 'this table is suspect' — and the reason for stopping at one hop disappears. Median mark age climbing is the signal that the current depth is already wrong.

Why it holds up over time. The underlying claim is durable: a signal that fires on everything conveys nothing. What will change is the depth the platform can afford, and it will change as lineage precision improves rather than as graph traversal gets faster.

Lesson. Precision and reach trade against each other in any propagation mechanism, and the failure mode of too much reach is indistinguishable from no mechanism at all. Publish the full picture; apply only the part people will still act on.

Credibility and debt

Keeping rules worth believing, keeping authority with the owner, and making every exception end.

ADR-12 · A bad rule cannot take down a pipeline: probation, circuit breaking and automatic demotion

Status: Accepted · Shown on views: 15, 05, 18

What protects the platform from its own assertions — a rule that crashes the engine, a rule that fires every night on a legitimate business change, a rule whose author has moved teams?

Context. The failure mode that kills a data quality programme is not missed defects; it is false positives. A rule that fires wrongly twice a week teaches everyone to route its alerts to a folder, and within a month the gate it feeds is advisory in practice while remaining mandatory on paper. Distributional assertions are the worst offenders, because a genuine business change — a new merchant category, a pricing experiment, a festival — looks exactly like a defect to a baseline trained on the previous month. There is also a cruder failure: an assertion whose plan reliably exceeds its budget or crashes the engine, which without isolation takes down the evaluation of every other assertion on that dataset.

Decision. Three structural protections. Per-assertion circuit breaking: an assertion that fails to execute for a threshold of consecutive runs is broken out individually, moved to probation, and its owner notified, while its siblings continue to be evaluated. Probation for every new assertion: thirty days as advise only, regardless of its declared action. Automatic demotion: an assertion whose false-positive rate exceeds 5% over a trailing thirty days is demoted to advise without waiting for a human, and raised as a debt item. Distributional assertions never leave probation automatically — promotion requires an explicit act after the rate has been measured.

How it is realised on Databricks and AWS. False positives are counted from acknowledged outcomes: a failing verdict that is overridden, dismissed, or followed by no upstream change within a window counts against the rate. The scheduler tracks consecutive execution failures per assertion version and trips the breaker in the work-item layer, so a poison assertion is skipped rather than retried into the ground. Demotion changes the assertion's effective action, never its declared one, so the author's intent stays visible next to the platform's judgement.

What it buys

• Alerts people still believe, which is the property that decides whether any of the rest of this is enforced.
• A poison assertion is isolated to itself rather than taking down evaluation of every other rule on that dataset.
• Demotion happens without depending on the attention of the people the noise has already exhausted.

What it costs

• A thirty-day delay before a new rule can protect anything.
• A false-positive metric inferred from imperfect signals — what people did after an alert, rather than what they meant.
• The real risk that a correct assertion firing on a genuinely broken upstream is demoted because nobody fixed the upstream, which is why demotion opens a debt item rather than closing the matter.

Choose differently when. If the false-positive inference proves too noisy to act on, the answer is an explicit acknowledgement workflow — someone marks a verdict as a false positive — rather than abandoning automatic demotion. The 5% threshold is an assumption; the mechanism is not.

Why it holds up over time. This is the decision that will matter most when assertions are proposed by models rather than typed by engineers, because the volume of plausible-but-wrong rules will go up by an order of magnitude. A platform that demotes on measured credibility scales to that; one that relies on review does not.

Lesson. Any detection system accumulates rules faster than it retires them, and its credibility is the resource that gets spent. Measure the credibility of each rule and act on it mechanically, because the humans who would have acted are the ones the noise has already worn out.

ADR-13 · Blocking authority belongs to the dataset owner, and the action is fixed at authoring time

Status: Accepted · Shown on views: 03, 13, 20

Who may declare that a failing assertion stops a pipeline — and is that declaration made when the rule is written or when it fires?

Context. Two questions that are usually conflated. On authority: an assertion that blocks can stop a Tier 1 dataset, so the power to write one is the power to halt another team's delivery. Granting it to any author is how a platform acquires an outage caused by a well-meaning engineer in a different domain; withholding it from everyone makes the platform advisory. On timing: deciding the action at run time, from the observed violation rate, adapts to reality — a rule could quarantine two rows and block on two thousand. It also means nobody can answer 'what will happen if this rule fails' before it fails, which makes the rule unreviewable, and puts the most consequential decision in the code path least able to be inspected.

Decision. The verdict action — advise, quarantine, block — is a property of the assertion and the dataset tier, declared at authoring time and refused by the tier policy where it is not permitted: no blocking on Tier 3, and no blocking on a distributional assertion alone. Only the dataset's owning team, or a delegate they name, may author a blocking assertion on their dataset, grant an override, or retire an assertion. Any author may propose a rule; the owner's approval is what lets it enforce. The single run-time adaptation is the quarantine ceiling in ADR-15, which is itself declared in advance.

How it is realised on Databricks and AWS. The tier and policy registry answers the authority question; the rule CI pipeline calls it before merge, so a rule declaring block on an ineligible dataset never lands. Owner approval is enforced through repository ownership on the rule DSL, and the same registry backs the override API's authorisation check at run time (view 20).

What it buys

• Every rule's consequence is stated before it can fire, so 'what happens if this fails' is a reviewable property.
• The team that can be paged is the team that authorised the block.
• A tier policy that refuses ineligible rules at merge rather than at 02:00.

What it costs

• Real friction for a cross-domain author who spots a defect and must get the owner's approval.
• Calibration of the tier policy matters: a high refusal rate is a platform signal, not an author problem.
• Adaptability is given up — a rule cannot decide at run time that this failure is worse than usual.

Choose differently when. If owner approval becomes a bottleneck that visibly suppresses coverage, the change worth making is a standing delegation for specific assertion types — schema and freshness, say — rather than opening blocking authority generally.

Why it holds up over time. Tying the authority to stop a pipeline to the accountability for that pipeline is durable because it is an organisational alignment, not a technical one. What will date is the mechanism — repository ownership today, something else later.

Lesson. Declare a control's consequence where it can be reviewed, not where it can be adapted. A system whose behaviour under failure is decided at failure time is a system nobody can sign off on.

ADR-16 · Every escape hatch is an expiring object that creates a debt item

Status: Accepted · Shown on views: 20, 15, 18

How does an override, a suppression or a demoted rule end — and who is accountable for it while it lasts?

Context. Quality debt does not accumulate because people are careless. It accumulates because an exception granted under pressure has no natural end: the block is overridden at 02:00 to get the report out, the incident closes, the override stays. Multiply by forty domains and two years and the platform has a set of unguarded datasets nobody can enumerate, each individually justified at the time. The same is true of suppressions granted during a migration and of assertions demoted for crying wolf. Represented as configuration flags, all three are invisible after the week they were created; represented as objects with owners and expiries, they become a backlog — which is uncomfortable, and correct.

Decision. Overrides, suppressions and probations are first-class objects with an owner, a justification, and a mandatory expiry; unbounded suppression is not supported by the API. Granting one creates a debt register entry in the same transaction, so there is no way to have the exception without the record. At expiry the state reverts automatically and the debt item ages rather than closing. Aged debt — an override past expiry, a suppression older than thirty days, a quarantine batch past retention, a demoted assertion never reworked, a Tier 1 dataset with no assertions — escalates to the owning team and appears on the domain scorecard.

How it is realised on Databricks and AWS. Overrides and debt items live in the same Aurora database and are written together; the override is recorded in the immutable audit log (ADR-14) before the state index changes. Expiry is enforced by the override service rather than by a consumer checking a timestamp, so the reversion happens whether or not anyone is looking. Scorecards are published per domain and per owner, because an unowned score is never paid down.

What it buys

• An enumerable set of current exceptions, which is something most platforms cannot produce at all.
• Automatic reversion at expiry, so an exception ends whether or not anyone is watching.
• A debt backlog that can be managed, aged and assigned rather than discovered during an audit.

What it costs

• Friction at the worst possible moment: during an incident, someone has to type a justification and choose an expiry.
• A scorecard that will look bad for the first few quarters, and will be read as a criticism of teams rather than of history.
• A visible debt register invites argument about its weighting — a better argument than the alternative, but an argument.

Choose differently when. Nothing technical flips this. The risk is erosion: a 'permanent exemption' field added for a genuinely awkward dataset, after which the mechanism is decorative. If such a case is unavoidable, it should be a tier change with the owner's sign-off — a statement about the dataset — not an exception with no end.

Why it holds up over time. Of the sixteen decisions, this is the one most likely to be quietly weakened and the hardest to notice if it is, because nothing breaks when an expiry becomes optional. It is also the one that does most to determine whether the platform still means anything in year three.

Lesson. Give every exception an owner and an end date at the moment it is created, because that is the only moment anyone will agree to either. A control with an unbounded escape hatch is a control with a schedule for becoming advisory.

Governance and data handling

Why a verdict must not leak what it judged, and why quarantine is custody rather than storage.

ADR-14 · A verdict must not become a side channel

Status: Accepted · Shown on views: 19, 20, 09

How does the platform make failures diagnosable without letting a verdict, its samples or its authorship reveal data the reader is not entitled to see?

Context. A useful failure message names the problem: 'the currency column contains INR for 412 rows in the EU partition'. A useful failure message is also a read of the data, delivered to whoever can see the verdict — which is a broader audience than whoever can see the table. The subtler version is authorship: an engineer who cannot read a salary column can write an assertion asserting salary values fall in a range, publish it, and learn from the pass or fail whether anyone earns above a threshold. Neither problem is exotic, and both are invisible in a design that treats the verdict as metadata rather than as a derived read of the data.

Decision. Four controls. Evaluation runs under a workload identity scoped to the assertion's registered targets, never a standing credential broader than their union. Catalogue access controls are enforced during evaluation, so an author who cannot read a column cannot publish an assertion over it. Sample violating values are redacted by default and emitted only for columns explicitly classified non-sensitive; everything else emits row identifiers rather than contents. And the override audit log — actor, justification, expiry, approver — is written to immutable storage before the state change it authorises, so the record cannot be the thing that failed.

How it is realised on Databricks and AWS. Unity Catalog grants are evaluated at assertion publication and again at execution, and the evaluator's identity is issued per assertion target set rather than held as a service-wide credential. The sample redactor sits in the evaluation plane, before any verdict crosses into the verdict store (view 19). The audit log is S3 Object Lock in compliance mode with a seven-year retention, which the platform's own operators cannot rewrite.

What it buys

• A verdict that can be shared widely without carrying the data it judged.
• An override trail that survives an adversarial question, because the platform's own operators cannot rewrite it.
• No standing full-estate credential, so compromising the quality service does not compromise the warehouse.

What it costs

• Harder debugging: engineers will ask for the failing values and the answer will often be row identifiers instead.
• A dependency on column classification being right, since redaction-by-default fails open for a column wrongly marked non-sensitive.
• Per-target identity issuance is more machinery than one service credential, and has to be operated.

Choose differently when. If classification coverage proves unreliable, the response is to narrow the exception further — sample values only for columns explicitly allow-listed by the data owner, rather than for anything not marked sensitive. That is a stricter default, not a different architecture.

Why it holds up over time. Treating derived signals as reads of the underlying data is a discipline that generalises to every observability surface — logs, metrics labels, error messages, model evaluations. The specific redaction rule will change; the recognition that a verdict is a read will not.

Lesson. Anything computed from sensitive data is sensitive until something deliberately makes it not. Failure messages, metric labels and test fixtures are the three places this is most often forgotten, and all three leak the same way.

ADR-15 · Quarantine only where violations are row-attributable, with a ceiling that escalates to block

Status: Accepted · Shown on views: 13, 09, 19

When is diverting bad rows the right answer, and at what point does diverting them become a way of hiding a broken upstream?

Context. Quarantine is the most attractive verdict action and the most easily abused. Where a violation is attributable to identifiable rows, diverting them and promoting the rest is genuinely better than blocking: the dataset stays usable, the bad rows are preserved with their context, and the fix is replayable. Two things go wrong. The first is category error — an aggregate failure, a row count out of bounds, a reconciliation gap against the ledger, is a property of the dataset, so there are no offending rows to divert and any attempt to quarantine 'the problem' is arbitrary. The second is scale: diverting two rows in a million is a correction, and diverting a quarter of a batch is a broken upstream being presented as partial success, with the remainder promoted and consumed as though it were complete.

Decision. Quarantine is available only to row-level and referential assertions, where a violation is attributable to identifiable rows. Aggregate, distributional and reconciliation failures may only advise or block. A per-dataset quarantine ceiling — 2% of rows in the batch by default — escalates the verdict to block when exceeded. Quarantined rows carry the full verdict context so a fix can be replayed, they inherit the source dataset's classification and masking policy, and re-admission happens only through the normal ingestion path. Rows still unfixed at the end of the retention window expire and are counted as quality debt.

How it is realised on Databricks and AWS. The diverter writes to a Delta quarantine table per source dataset, partitioned by source dataset and date, with assertion id and version, source version, partition and timestamp as columns. Classification and masking are inherited from the source in Unity Catalog rather than configured separately, so quarantine cannot become a lower-governance copy. Expired batches are reported by volume into the debt register (ADR-16) rather than being silently dropped.

What it buys

• Datasets stay usable in the common case, where a handful of rows are wrong and the rest are fine.
• Quarantine cannot disguise a systemic break as partial success, because the ceiling escalates to block.
• Quarantined data inherits the source's classification, so an automated process cannot downgrade governance.

What it costs

• A ceiling per dataset that will be argued about, and that is a judgement call rather than a derived number.
• A quarantine estate that has to be governed, aged and reported on, in tables no domain team naturally owns.
• Re-admission through ingestion is slower than a direct write — deliberately, and it will still be complained about.

Choose differently when. If a legitimate workload routinely produces more than 2% violating rows — a genuinely noisy third-party feed where the noise is expected — the answer is a dataset-specific ceiling with the owner's sign-off, not the removal of the ceiling. A rising rate of ceiling escalations across a domain is a producer-quality signal, not a threshold problem.

Why it holds up over time. The attribution rule is durable because it follows from the assertion type rather than from policy: an aggregate failure has no rows to divert in any decade. The ceiling is durable as a concept — a correction mechanism must have a point at which it admits the problem is systemic.

Lesson. Any mechanism that partially accepts bad input needs a threshold beyond which it stops pretending, and the threshold should trigger escalation rather than a bigger version of the same accommodation.

Every package used, in one table

Twelve terms this package uses precisely. Where a definition has a load-bearing alternative — a way the term is commonly used that would produce a different architecture — the alternative is named.