intermediate
2 min answer
Which automated quality gates are worth blocking a release on, and which are ceremony?
Show the full answer Hide the answer
Worth gating on
- Tests passing, where the suite is trustworthy — which is a precondition rather than an assumption.
- Build reproducibility and provenance verification, checking the artefact came from the expected source and builder.
- Deterministic policy checks: no privileged containers, no publicly readable storage, no unencrypted data at rest, required ownership metadata. These are never wrong in a way that erodes trust, which is precisely what makes them enforceable.
- Contract verification against every consumer, which converts "we will find out in production" into "the build fails and names the consumer."
- Destructive schema changes blocked unless they have gone through expand-contract.
- Known-exploitable vulnerabilities in reachable code paths — with reachability being the operative word.
- The segmented comparative canary gate, which catches more real defects than everything above combined.
Ceremony
- Code coverage thresholds. Coverage measures execution, not verification, and once it is a target it is met by tests that execute code without asserting anything. It is a useful diagnostic — a module at 0% is informative — and a poor gate.
- Static analysis with a large ignored backlog, which trains everyone to ignore the output.
- Vulnerability scanning without reachability analysis, producing thousands of findings mostly in code paths never executed — the same dynamic as alert fatigue, with the same consequence.
- Manual sign-off from someone without the context to assess it, where the approval is granted by default and the delay is real while the risk reduction is not.
- A change advisory board reviewing changes it cannot meaningfully evaluate.
- Manual test passes duplicating automated coverage.
The distinguishing test
Does this gate ever actually block anything, and when it blocks, is it right?
A gate with a 100% approval rate is a delay, not a control. A gate that is frequently wrong trains everyone to override it, after which it is a delay too.
The design rules
- Automate what can be checked deterministically; leave the rest to judgement applied earlier.
- Move human judgement to design time, where it changes the outcome, rather than to release time, where it can only approve or obstruct.
- Every gate needs a documented override with a named approver, because a gate with no override is deleted the first time it is genuinely wrong — and a visible exception is far better than an invisible workaround.
- Track the override rate, which is the honest signal: a gate overridden routinely should become a guardrail or have its rule fixed.
- Prefer generated compliance to checked compliance. A golden path that produces the correct configuration removes the need for the gate entirely — a platform relying mainly on gates has not made the right thing easy, it has made the wrong thing difficult, which is felt as bureaucracy and is a weaker position.