Warn-Mode Debt

A policy engine is introduced in warn mode so that nobody is blocked while teams adapt. Eighteen months later it covers 400 repositories and reports 2,300 open violations, and leadership asks for it to be switched to blocking.

The count is not a measure of risk. It is a measure of how long the gate has been advisory, because a warning that never stops anything is read once and filtered thereafter.

Switching straight to blocking fails twice: every pipeline with an existing violation breaks including the ones fixing something urgent, and the rules that were never enforced turn out to contain false positives written against an idealised repository layout. The second failure is the durable one, because a policy engine that blocks on nonsense loses its credibility for about a year.

Why it matters

Warn mode feels like a safe rollout and is actually a decision to defer enforcement indefinitely. The cost compounds: violations accrue at whatever rate the organisation creates them, and the political cost of enforcement rises with the count.

It also hides rule quality. Enforcement is what tests a rule against reality; until then, nobody knows which rules are right, and the backlog mixes genuine findings with artefacts in unknown proportion.

Implementation patterns

• Triage the rules, not the violations. Per policy: how many violations, how many plausibly false, what harm the rule prevents. A small number of rules usually produces most of the count.
• Delete or fix the wrong rules. A rule with 600 violations and no incident behind it is a bad rule, and removing it is a legitimate outcome rather than a defeat.
• Block on new violations only. Enforce for changed files or new resources, so the existing population is grandfathered and the number stops growing immediately. This is usually available within a sprint and is the single highest-value step.
• Burn down by harm, with owners and dates, tracked as work rather than as a dashboard.
• Flip per rule as each backlog clears, never all at once.
• Keep a time-boxed waiver path with an owner and an expiry, because an unavailable exception path produces a disabled gate.

Industry example

The pattern is the same one type checkers and linters went through: gradual typing, which arrived in Python 3.5 in 2015, and --warn-only modes exist because a codebase cannot be made compliant in one step, and the projects that succeeded all did the same thing — enforce strictly on new code, grandfather the old, and burn down by module. Policy-as-code tooling adopted the same staging vocabulary as it spread through cloud estates in the 2020s, and organisations that skipped the grandfathering step reliably ended up reverting.

Failure scenarios

• A blanket waiver issued at the flip, which looks like compliance and enforces nothing.
• Reverting the gate, after which the next attempt is harder because the engine now has a reputation.
• Auto-fixing everything, which is fine for formatting and dangerous where the violation reflects a design decision nobody has reviewed.
• A deadline announced without triage, so the first false positive at the deadline costs the policy its authority.
• Rules added in warn mode forever, so the engine accumulates a second backlog behind the first.

Trade-offs

Blocking on new violations only leaves the legacy population non-compliant, which an auditor may count as a finding; enforcing everywhere at once buys full compliance and costs delivery and credibility. The middle path is honest and has to be stated as a plan with dates, because "blocking for new code" without a burndown is just warn mode with better marketing.

When not to use it

A brand-new rule with zero existing violations should be blocking from day one. That is the whole argument for enforcing at introduction: no backlog, no negotiation, no grandfathering. Warn mode is justified for a genuinely new rule whose false-positive rate is unknown — for a week or two, with a stated date on which it becomes blocking — and never as an open-ended state.

Interview question

Q: You have inherited a policy engine in warn mode with 2,300 violations across 400 repositories, and a mandate to make it blocking this quarter. What is your plan, and what do you tell leadership about what "blocking" will mean in practice by the end of it?

What a strong answer covers: triaging rules before violations; deleting bad rules as a legitimate outcome; blocking on new violations as the first enforcement step; a burndown by harm with named owners; flipping per rule rather than globally; a time-boxed waiver path; and telling leadership plainly that the quarter ends with new violations blocked and a dated burndown for the rest, rather than promising full enforcement and delivering a blanket waiver.

Quick check

Quiz: Why is the violation count of a warn-mode gate a poor measure of risk? — It measures how long the gate has been advisory, and it mixes genuine findings with untested false positives in unknown proportion.

Flashcard: What is the first enforcement step that costs almost nothing? — Block on new violations only, so the backlog is grandfathered and stops growing from that day.