Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
60 results for “Testing in Production”
Synthetic Monitoring
Continuously executing a real user journey against production from outside it, so a broken journey is detected before a user reports it.
Contract Testing
Verifying that a provider satisfies the expectations each of its consumers actually relies on, without running all the services together.
Failure Injection Testing
Deliberately introducing faults into a system under test to verify that timeouts, retries, fallbacks and circuit breakers behave as designed.
Load Testing
Driving a system with realistic traffic at a target volume to verify it meets its performance targets before real users do.
Production Parity Gap
The enumerated list of ways a pre-production environment differs from production, which is the list of defect classes it cannot catch.
Testing Strategy Shape
The distribution of tests across levels, chosen so that feedback is fast where it can be and confidence is real where it must be.
Testing Trophy
A distribution weighted towards integration tests rather than unit tests, appropriate where most of the risk lives at boundaries rather than in logic.
Adversarial Evaluation
Deliberately attempting to make a model behave badly, because a probabilistic system with no fixed expected output cannot be verified by conventional testing.
Automated Accessibility Coverage
The proportion of accessibility criteria a tool can decide mechanically — around a third — and the explicit acknowledgement that the rest needs people.
Blue-Green Deployment
Running two identical production environments and switching traffic from the old one to the new one in a single cut, with the old kept warm for rollback.
Booking.com's Experimentation Platform
Booking.com runs over a thousand concurrent experiments and treats the ability to test any change safely as a platform capability rather than a product feature.
Change Failure Rate
The proportion of deployments that cause a production failure requiring remediation, and the DORA metric that keeps the others honest.
Chaos Engineering
Deliberately injecting failure into a system to discover, before an incident does, which of your resilience assumptions are false.
Consumer-Driven Contract
A contract derived from what consumers actually use, published to the provider, and verified in the provider's own build.
Continuous Profiling
Sampling CPU, memory and lock profiles from production continuously at low overhead, so resource usage can be attributed to specific code paths.
Contract Verification Gate
A provider's pipeline stage that replays every consumer's recorded expectations and fails the build if any would break.
Deployment Pipeline
The automated path from commit to production, structured so that each stage increases confidence and the whole is fast enough to be run on every change.
Fitness Function
An automated check that an architectural characteristic still holds, run continuously rather than reviewed occasionally.
Game Day
A scheduled exercise in which a failure is deliberately introduced and the team responds as though it were real, to test the system and the response together.
Golden Path
One opinionated end-to-end route from idea to production that is genuinely easier than the alternatives, and is supported rather than mandated.
Independent Deployability
The property that a service can be released to production without coordinated release of any other, which is the defining benefit of microservices.
Your SRE team wants to run fault injection in production. Leadership is nervous. How do you make the case and what do you insist on?
The case Failures happen in production whether or not they are injected. The choice is between discovering how the system responds at three in the morning durin
Your load tests pass consistently and production still falls over at peak. What is wrong with the tests?
The workload model is almost certainly unrealistic Data distribution is the usual culprit. Synthetic tests spread requests uniformly across keys, which produces
A serverless API works in testing and fails under load with connection errors. The database is at 5% CPU. Explain and fix.
The mechanism Serverless functions scale by creating independent execution environments , each with its own process and its own connection pool. Two hundred con
Delivery leadership says releases are too slow and wants the engineering team to "move faster". Lead time from commit to production is 21 days. How do you investigate, and what do you expect to find?
Do not accept the framing "Move faster" assumes the constraint is how fast code is written. That is almost never where the twenty one days went, and starting an
Half your production changes are made through the console during incidents. The IaC repository no longer matches reality. What do you do?
Why this matters beyond tidiness The code has stopped being a description of reality, which removes the properties it was adopted for: the ability to recreate a
Partner integrations take four months from contract signature to production traffic. Engineering says the work is three days. Where does the time go and how do you cut it?
Map the elapsed time honestly The engineering estimate is correct and irrelevant. The four months are: Legal and contractual review. Security questionnaires and
Production is degraded after a release. The team tries to roll back and discovers a migration has already run. What do you do now, and what do you change afterwards?
Now: stop trying to roll back Reverting the deployment would run the previous code against a schema it does not know, which is a second and less understood fail
Three teams keep breaking each other's services with API changes. Someone proposes contract testing. How do you introduce it without a six-month programme?
Start with one pair, not with the estate Pick the pair of services that has broken each other most recently. Instrument that one relationship end to end: the co
A brief database slowdown caused a two-hour full outage. Explain the likely amplification chain and the fixes at each stage.
The chain Slow database → queries that took 10 ms take 2 s. Thread pool exhaustion. Requests hold threads for 200× longer. A pool of 50 that handled the load ea
A team's CI suite fails roughly one run in three for reasons unrelated to the change. Everyone reruns until green. How do you recover the situation?
Recognise what has actually been lost The suite is no longer a gate. Once the team's reflex on red is "rerun", that reflex is applied to genuine failures too, a
An end-to-end suite of 340 tests takes four hours and fails spuriously about half the time. The team wants to parallelise it. Is that the right move?
Parallelising treats the symptom It might halve the runtime. It will not touch the flakiness — in fact parallelisation often worsens it, by exposing shared stat
End-to-end tests fail intermittently and nobody owns them. QA says the developers broke them; developers say the tests are flaky. How do you resolve this?
The ownership gap is the actual problem A test suite owned by nobody is maintained by nobody, and each failure becomes a negotiation rather than a fix. That is
Integration failures between 30 services are found in a shared staging environment, days after merge. Propose a change.
Why the current model fails A shared staging environment is a serialised, high latency feedback channel . Failures are found late, attribution is ambiguous (who
Testing in Production
Synthetic transactions, dark launches and shadow traffic, done deliberately and safely.
Environment Parity
The differences between staging and production that decide which bugs survive to release.
Test Data Management
Realistic data without copying production personal data into a weaker environment.
Accessibility Testing
Automated checks, their ceiling, and the manual testing that has to sit above it.
Contract Testing
Verifying what consumers actually rely on, without a shared environment.
Contract Testing at Scale
Keeping dozens of services compatible without an environment that runs all of them.
DR Testing
Restore drills, timed against the stated RTO, into a clean environment.
Load Testing
Realistic data, realistic mix, and a ramp rather than a step.
Mutation Testing
Measuring whether tests would actually notice a defect, not just cover a line.
Resilience Testing
Exercising retries, breakers and fallbacks that are otherwise never run.
Security Testing in the Pipeline
SAST, DAST, dependency and secret scanning, and what to do with the findings.
Soak Testing
Long runs that surface leaks and slow degradation.
Stress Testing
Pushing past target to learn what breaks first and how it fails.
Testing & Quality Architecture
General material on designing a testing strategy as an architectural concern.
Testing Strategies
The pyramid, and the contract tests distributed systems add to it.
Chaos as a Test
Fault injection with a hypothesis, a blast radius and an abort condition.
Consumer-Driven Contracts
Consumers declaring what they rely on, and providers verifying against those declarations.
Continuous Controls Monitoring
Testing controls continuously instead of sampling them once a year.
Delivery & Release Engineering
General material on getting a change from commit to production safely and often.
End-to-End Test Economics
Why broad end-to-end suites get slow, flaky and abandoned, and what to keep.
Flaky Test Management
Quarantine, detection, and the trust a suite loses once red stops meaning broken.
Game Days
Testing the response — runbooks, access, comms — not only the system.
Integration Test Boundaries
What sits inside a test's boundary, what is faked, and the confidence that follows.