intermediate 2 min answer

Which automated quality gates are worth blocking a release on, and which are ceremony?

quality-gatescoveragestatic-analysiscigoodhart
Show the full answer Hide the answer

Worth gating on

  • Tests passing, where the suite is trustworthy — which is a precondition rather than an assumption.
  • Build reproducibility and provenance verification, checking the artefact came from the expected source and builder.
  • Deterministic policy checks: no privileged containers, no publicly readable storage, no unencrypted data at rest, required ownership metadata. These are never wrong in a way that erodes trust, which is precisely what makes them enforceable.
  • Contract verification against every consumer, which converts "we will find out in production" into "the build fails and names the consumer."
  • Destructive schema changes blocked unless they have gone through expand-contract.
  • Known-exploitable vulnerabilities in reachable code paths — with reachability being the operative word.
  • The segmented comparative canary gate, which catches more real defects than everything above combined.

Ceremony

  • Code coverage thresholds. Coverage measures execution, not verification, and once it is a target it is met by tests that execute code without asserting anything. It is a useful diagnostic — a module at 0% is informative — and a poor gate.
  • Static analysis with a large ignored backlog, which trains everyone to ignore the output.
  • Vulnerability scanning without reachability analysis, producing thousands of findings mostly in code paths never executed — the same dynamic as alert fatigue, with the same consequence.
  • Manual sign-off from someone without the context to assess it, where the approval is granted by default and the delay is real while the risk reduction is not.
  • A change advisory board reviewing changes it cannot meaningfully evaluate.
  • Manual test passes duplicating automated coverage.

The distinguishing test

Does this gate ever actually block anything, and when it blocks, is it right?

A gate with a 100% approval rate is a delay, not a control. A gate that is frequently wrong trains everyone to override it, after which it is a delay too.

The design rules

  • Automate what can be checked deterministically; leave the rest to judgement applied earlier.
  • Move human judgement to design time, where it changes the outcome, rather than to release time, where it can only approve or obstruct.
  • Every gate needs a documented override with a named approver, because a gate with no override is deleted the first time it is genuinely wrong — and a visible exception is far better than an invisible workaround.
  • Track the override rate, which is the honest signal: a gate overridden routinely should become a guardrail or have its rule fixed.
  • Prefer generated compliance to checked compliance. A golden path that produces the correct configuration removes the need for the gate entirely — a platform relying mainly on gates has not made the right thing easy, it has made the wrong thing difficult, which is felt as bureaucracy and is a weaker position.