advanced 3 min answer

An organisation with hundreds of services finds integration bugs in production despite extensive testing. How do consumer-driven contract tests fit, and what do they not solve?

contract-testingpactintegrationciboundaries
Show the full answer Hide the answer

Why the bugs get through

Each service is tested in isolation against mocks that the service's own team wrote. The mock encodes the team's belief about the dependency's behaviour, and that belief drifts from reality without anyone noticing, because nothing verifies the mock.

The alternative that teams reach for — a shared integration environment with everything deployed — has its own well-known problems: it is slow, it is perpetually broken, failures are hard to attribute, and it becomes a queue. At hundreds of services it is not a viable gate.

What contract testing does

  • The consumer writes its expectations as a contract: for this request, I expect a response with these fields and these types. This is derived from the consumer's actual usage, so it captures what is genuinely needed rather than the whole interface.
  • The contract is published to a shared broker.
  • The provider verifies every consumer's contract in its own pipeline. A change that breaks any consumer fails the provider's build, before deployment, with a specific message naming the consumer and the field.

The key properties: no shared environment, no deployment ordering, fast feedback in the pipeline of the team making the change, and — crucially — the provider learns what is actually used. A field nobody's contract mentions can be removed safely, which is otherwise unknowable.

What it does not solve

  • Semantic correctness. The contract verifies shape, not meaning. A provider that returns amounts in cents instead of dollars satisfies every contract and breaks every consumer. This is the most important limitation and it is routinely forgotten.
  • Behavioural sequences. "Create then read returns what you wrote" is not expressible in a request/response contract.
  • Non-functional characteristics. Latency, throughput and error rates are untouched.
  • Emergent and cross-service behaviour, including anything involving asynchronous flows across several services.
  • Consumers who do not participate. Contract testing covers the consumers who wrote contracts; external and third-party consumers are invisible, which is why public APIs still need versioning discipline.
  • Data-dependent behaviour, where the provider's response depends on state the contract does not establish.

Making it work at scale

  • Contracts generated from real consumer tests, not hand-written, or they drift from actual usage.
  • The broker as the integration point, with can-i-deploy checks gating deployment on all consumers' contracts being satisfied by the version about to ship.
  • Versioned contracts tied to deployed versions, so the provider verifies against what is actually running in each environment — not against the latest contract from a consumer that has not deployed yet.
  • A small number of end-to-end tests retained for the critical journeys, covering the semantic and sequential gaps. Contract tests replace most integration tests, not all of them, and the residual set should be deliberately chosen and small.
  • Ownership discipline: a failing contract verification is the provider's signal to talk to the consumer, not to delete the contract.

The honest summary

Contract testing converts "we will find out in production" into "the build fails and names the consumer." That is a large improvement and it is not a guarantee of correctness — it verifies that both sides agree about the shape of the conversation, not that either side is right about its meaning.