Search the practice set
275 questions, 991 terms and 600 topics in 30 areas.
60 results for “End-to-End Test Economics”
End-to-End Test Cost Curve
How the cost of a broad end-to-end suite grows with its size while its marginal value falls, and where the two cross.
Test Pyramid
A distribution of tests weighted towards many fast unit tests, fewer integration tests, and very few slow end-to-end tests.
Boundary Volatility Test
Evaluating a proposed service boundary by asking whether the things on either side change for different reasons and at different rates.
Cache Hit Ratio Economics
The non-linear relationship between cache hit rate and backend load, which determines whether a cache improvement is worth making.
Control Test Automation
Executing a control's test continuously against the whole population rather than sampling it annually, which changes both the detection latency and the strength of the evidence.
Differentiation Test
The question of whether a capability is a source of competitive advantage, used as the primary filter in build-versus-buy decisions.
Refactoring Under Test
Changing internal structure without changing behaviour, with tests as the mechanism that makes the claim verifiable.
Soak Test
Running sustained realistic load for hours or days to expose defects that accumulate over time rather than appearing under peak load.
Stress Test
Driving load beyond expected capacity to observe how the system behaves at and past its breaking point.
Test Data Provisioning
Getting each test the data it needs, in a state it can rely on, without copying production personal data into a weaker environment.
Test Double Boundary
The line inside a test between what is real and what is substituted, which determines exactly what the test can and cannot prove.
Test Quarantine
Moving an intermittently failing test out of the blocking suite into a tracked, owned backlog, so a red build keeps meaning something.
Test Strategy Altitude
Deciding which risks are verified at which level, so that each layer tests something the layers below it structurally cannot.
Unit Economics
Infrastructure cost expressed per unit of business value delivered, which reveals efficiency trends that absolute spend cannot.
Arrival Rate Model
Driving a load test by requests arriving per second regardless of how the system responds, rather than by a fixed number of virtual users.
Assumption Excavation
Deliberately surfacing the unstated beliefs behind a design or a requirement, to test which are constraints and which are merely habits.
Booking.com's Experimentation Platform
Booking.com runs over a thousand concurrent experiments and treats the ability to test any change safely as a platform capability rather than a product feature.
Bottom Line Up Front
Placing the conclusion and the requested action in the first lines, on the assumption that most readers will not reach the end.
Chaos Experiment
A controlled test that injects a specific failure to verify a hypothesis about the system's resilience, with a defined blast radius and abort condition.
Context Propagation
Carrying request-scoped identifiers and metadata across every service, thread and asynchronous boundary so a single flow remains traceable end to end.
Cost per Request
The fully-loaded infrastructure cost of serving one request, and the unit that makes architectural cost decisions comparable.
Data Retention Policy
A defined rule for how long each class of data is kept, where, and what happens at the end of it.
Deprecation Window
The stated period between announcing that an internal interface will be removed and removing it, with the removal actually happening at the end.
Developer Portal
The self-service surface where consumers discover APIs, read documentation, obtain credentials and test calls — the main determinant of adoption.
Failure Injection Testing
Deliberately introducing faults into a system under test to verify that timeouts, retries, fallbacks and circuit breakers behave as designed.
Game Day
A scheduled exercise in which a failure is deliberately introduced and the team responds as though it were real, to test the system and the response together.
Golden Path
One opinionated end-to-end route from idea to production that is genuinely easier than the alternatives, and is supported rather than mandated.
Half-Open State
The circuit breaker state that allows a limited number of trial requests through to test whether a failed dependency has recovered.
Idempotent Sink
A destination where writing the same record twice has the same effect as writing it once, which is what makes end-to-end exactly-once achievable at all.
Mutation Score
The proportion of deliberately introduced faults that the test suite detects — a measure of whether tests would notice a defect, unlike coverage.
Non-Functional Acceptance
Treating quality-attribute targets as acceptance criteria with automated verification, so a release can fail on latency the way it fails on a broken feature.
An end-to-end suite of 340 tests takes four hours and fails spuriously about half the time. The team wants to parallelise it. Is that the right move?
Parallelising treats the symptom It might halve the runtime. It will not touch the flakiness — in fact parallelisation often worsens it, by exposing shared stat
End-to-end tests fail intermittently and nobody owns them. QA says the developers broke them; developers say the tests are flaky. How do you resolve this?
The ownership gap is the actual problem A test suite owned by nobody is maintained by nobody, and each failure becomes a negotiation rather than a fix. That is
Netflix built its own CDN; Dropbox moved storage off S3. Both are usually wrong. What conditions made them right, and how do you test for those conditions?
What the interviewer is testing Whether you can extract the conditions from a famous decision rather than the decision itself. These two cases are the most comm
A DR test fails: the secondary region cannot launch enough instances. What happened, and what standing checks prevent it?
What happened Service quotas in the secondary region are far lower than in the primary , because nothing has ever run there at scale. Quotas are per account and
A front-end lead proposes micro-frontends so four teams can deploy independently. The application is a single React SPA. What is your assessment?
Test the premise first The stated goal is independent deployment. Ask what currently prevents it, because the answer is frequently not the architecture. If the
A front-end team wants to replace their global state library because "state management is unmanageable". How do you evaluate the request?
Ask what is actually in the store The complaint almost always resolves to one distinction not having been made: server state versus UI state . Server state is d
A payments API must guarantee that a network retry never charges a customer twice. Design the mechanism end to end.
Why this is unavoidable A client that times out on a POST cannot know whether the request succeeded. Not retrying risks a lost payment; retrying risks a duplica
Six teams share one integration test environment. Bookings are made a week ahead and releases slip when someone overruns. How do you fix it?
Name the cost first The queue is not an inconvenience; it is lead time. Measure it: for the last twenty changes, how many days elapsed between "ready to test" a
When should a service call another synchronously, and when should it publish an event instead? Give me the deciding test, not a preference.
The deciding test Does this user action succeed or fail based on this callee's response? If yes, the call is synchronous, because you need the answer to decide.
Your test quarantine has grown to 140 tests over a year. What has gone wrong and how do you recover?
What went wrong Quarantine without a cap and without a deadline becomes a graveyard. Each individual decision was reasonable — move the flaky test aside, raise
A team's CI suite fails roughly one run in three for reasons unrelated to the change. Everyone reruns until green. How do you recover the situation?
Recognise what has actually been lost The suite is no longer a gate. Once the team's reflex on red is "rerun", that reflex is applied to genuine failures too, a
Cloud spend has grown 60% in a year while revenue grew 15%. The CFO wants a 30% reduction. How do you respond?
Change the metric before accepting the target Absolute spend is the wrong frame. Cost per unit of business value — per order, per active user, per gigabyte proc
End-to-End Test Economics
Why broad end-to-end suites get slow, flaky and abandoned, and what to keep.
Chaos as a Test
Fault injection with a hypothesis, a blast radius and an abort condition.
Flaky Test Management
Quarantine, detection, and the trust a suite loses once red stops meaning broken.
Integration Test Boundaries
What sits inside a test's boundary, what is faked, and the confidence that follows.
Non-Functional Test Strategy
Testing availability, latency, security and recovery rather than only behaviour.
Performance Test Design
Workload models, warm-up, think time, and the distribution the average hides.
Test Architecture Strategy
Choosing what to verify where, given the failure modes that actually occur.
Test Data Management
Realistic data without copying production personal data into a weaker environment.
Test Pyramid Shapes
Pyramid, trophy and honeycomb, and the system properties that justify each shape.
Unit Economics
Cost per request, per tenant, per transaction — the actionable number.
Continuous Integration Discipline
Integrating to the mainline daily, and the test speed and review culture that requires.
Cost & FinOps
General material on the economics of an architecture.
Data Platform Architecture
General material on designing the analytical data estate end to end.
First-Principles Reasoning
Reducing to physics, mathematics and economics, then reasoning up.
Message Formats
JSON, Protobuf, Avro — schema evolution and payload economics.