Environment Parity
The differences between staging and production that decide which bugs survive to release.
4 to work through
-
intermediate
A change passes every test in staging and fails immediately in production. Staging is a faithful copy of the topology. What are the most likely causes, and what would you change?
2 min answer -
intermediate
Which dimensions of environment parity actually matter, and which are safe to differ?
1 min answer -
advanced
A platform has already been through the failure Fastly published in 2021 - a defect shipped on 12 May that a customer's valid configuration armed on 8 June - and has since implemented per-point-of-presence staged activation of customer configurations plus grammar-based fuzzing of the configuration surface. Six months later the same class recurs in a different subsystem. Which gap did that action list never close?
3 min answer -
advanced
On 14 March 2023 Reddit upgraded a Kubernetes cluster from 1.23 to 1.24 after the same upgrade had already succeeded elsewhere, and was down for 314 minutes. The upgrade removed a node label that a hand-configured Calico route reflector selected on. Which parity assumption failed?
2 min answer
4 terms in this topic
Configuration Corpus Parity
How closely the set of configurations a pre-production environment exercises resembles the set production actually carries - the parity dimension tha…
conceptEnvironment Parity
How closely a pre-production environment reproduces the properties of production that actually cause failures.
conceptProduction Parity Gap
The enumerated list of ways a pre-production environment differs from production, which is the list of defect classes it cannot catch.
conceptProvenance Parity
Similarity between environments in how they were built and changed over time - the parity dimension that decides whether a rehearsal transfers, and t…
Neighbouring topics
Testing & Quality Architecture
General material on designing a testing strategy as an architectural concern.
Test Architecture Strategy
Choosing what to verify where, given the failure modes that actually occur.
Test Pyramid Shapes
Pyramid, trophy and honeycomb, and the system properties that justify each shape.
Integration Test Boundaries
What sits inside a test's boundary, what is faked, and the confidence that follows.
Contract Testing at Scale
Keeping dozens of services compatible without an environment that runs all of them.
Consumer-Driven Contracts
Consumers declaring what they rely on, and providers verifying against those declarations.
Test Data Management
Realistic data without copying production personal data into a weaker environment.
Synthetic Data
Generating data with the shape and edge cases of the real thing, and where it misleads.
Service Virtualisation
Standing in for a dependency you cannot call, and keeping the stand-in honest.
End-to-End Test Economics
Why broad end-to-end suites get slow, flaky and abandoned, and what to keep.
Non-Functional Test Strategy
Testing availability, latency, security and recovery rather than only behaviour.
Performance Test Design
Workload models, warm-up, think time, and the distribution the average hides.
Chaos as a Test
Fault injection with a hypothesis, a blast radius and an abort condition.
Security Testing in the Pipeline
SAST, DAST, dependency and secret scanning, and what to do with the findings.
Accessibility Testing
Automated checks, their ceiling, and the manual testing that has to sit above it.
Mutation Testing
Measuring whether tests would actually notice a defect, not just cover a line.
Flaky Test Management
Quarantine, detection, and the trust a suite loses once red stops meaning broken.
Testing in Production
Synthetic transactions, dark launches and shadow traffic, done deliberately and safely.
Quality Gates
Thresholds that block a release, who may override them, and how they decay.