Quality Gates
Thresholds that block a release, who may override them, and how they decay.
5 to work through
-
intermediate
A 12-year-old service has 4% test coverage and no tests for its core logic. It changes twice a year. Do you apply your quality gates to it?
2 min answer -
intermediate
An organisation's quality gates have accumulated and now block releases without reducing defects. How should the gate set be rationalised?
2 min answer -
intermediate
Leadership mandates 80% code coverage across all services. What do you say?
2 min answer -
intermediate
Which automated quality gates are worth blocking a release on, and which are ceremony?
2 min answer -
advanced
A security vendor's release process gates sensor code behind staged rollout with canaries, and ships detection content through a separate path validated by a content checker and pushed to every customer within minutes. In July 2024 CrowdStrike's own root cause analysis describes exactly this shape, and a mismatch between 21 fields supplied and 20 fields read produced an out-of-bounds read that crashed around 8.5 million Windows hosts. Review the process. What would you remove, what would you add, and what would you leave alone?
3 min answer
4 terms in this topic
Change Failure Rate
The proportion of deployments to production that cause a degradation requiring remediation.
conceptContent-Channel Release Path
Any route by which an artefact reaches a running system without passing the gates that code passes - remote config, feature flags, rules tables, mode…
practiceDefinition of Done
The agreed, explicit conditions under which work is finished, which functions as a quality gate only when it is automated and non-negotiable.
practiceQuality Gate Threshold
The specific numeric bar a change must clear, and the perverse incentives it creates when chosen carelessly.
Neighbouring topics
Testing & Quality Architecture
General material on designing a testing strategy as an architectural concern.
Test Architecture Strategy
Choosing what to verify where, given the failure modes that actually occur.
Test Pyramid Shapes
Pyramid, trophy and honeycomb, and the system properties that justify each shape.
Integration Test Boundaries
What sits inside a test's boundary, what is faked, and the confidence that follows.
Contract Testing at Scale
Keeping dozens of services compatible without an environment that runs all of them.
Consumer-Driven Contracts
Consumers declaring what they rely on, and providers verifying against those declarations.
Test Data Management
Realistic data without copying production personal data into a weaker environment.
Synthetic Data
Generating data with the shape and edge cases of the real thing, and where it misleads.
Environment Parity
The differences between staging and production that decide which bugs survive to release.
Service Virtualisation
Standing in for a dependency you cannot call, and keeping the stand-in honest.
End-to-End Test Economics
Why broad end-to-end suites get slow, flaky and abandoned, and what to keep.
Non-Functional Test Strategy
Testing availability, latency, security and recovery rather than only behaviour.
Performance Test Design
Workload models, warm-up, think time, and the distribution the average hides.
Chaos as a Test
Fault injection with a hypothesis, a blast radius and an abort condition.
Security Testing in the Pipeline
SAST, DAST, dependency and secret scanning, and what to do with the findings.
Accessibility Testing
Automated checks, their ceiling, and the manual testing that has to sit above it.
Mutation Testing
Measuring whether tests would actually notice a defect, not just cover a line.
Flaky Test Management
Quarantine, detection, and the trust a suite loses once red stops meaning broken.
Testing in Production
Synthetic transactions, dark launches and shadow traffic, done deliberately and safely.