Verification Fan-Out

Contract testing is usually adopted for a correctness reason - catch an integration break without a shared environment - and abandoned for an arithmetic one. Somewhere past a few dozen consumers, verification becomes the longest stage in the pipeline and teams start skipping it.

Verification fan-out predicts when that happens. It counts executions rather than contracts: how many times a provider's verification suite runs per day, and against how many contracts each time. The second factor is where the cost curve bends.

Why it matters

The arithmetic is unambiguous. With 40 consumers each publishing on 6 merges a day there are 240 publications. If every publication triggers a full verification of all 40 recorded contracts and a run takes 90 seconds, that is 240 x 90 s, about 6 hours of compute a day - affordable, and quadratic in consumer count. At 400 consumers the same pattern gives roughly 585 hours a day, two dozen machines re-checking contracts that did not change.

With result caching keyed on (provider version, contract content hash) the work becomes linear: each publication verifies its own contract in about 2 seconds, so the day is under 10 minutes. The same practice is either trivial or impossible depending on one cache, which is why this belongs in a design review rather than a retrospective.

Implementation patterns

• Content-hash every contract and skip pairs already verified. Most publications are byte-identical to the last, which removes 90% or more of the executions.
• Verify against deployed versions, not all history. Record which consumer versions run in each environment and verify against those; a provider checked against every contract ever recorded is blocked by consumers nobody runs. Pact's broker does this with environment records and its can-i-deploy query.
• Separate the gate from the batch: on a pull request verify consumers that are in production, and run the full matrix nightly.
• Watch publication-to-answer latency, not just compute - how long a consumer waits for a verdict before it can deploy.
• Keep the run fast enough to execute on demand. If verification needs a seeded database and 11 minutes, fix the run before optimising the matrix.
• Alert on fan-out growth, which climbs silently as teams who never see the provider's pipeline add consumers.

Industry example

The open-source Pact tooling is the clearest documented treatment, because its broker exists largely to manage fan-out: contracts are published with a content identifier, verification results are recorded against the pair of versions, and deployment is gated by a can-i-deploy query that reads recorded results rather than re-running tests. Published adoption accounts from large marketplaces describe arriving here the same way: integration environments were unstable because of external dependencies, which makes broad end-to-end testing brittle, so verification moved to contracts - and the fan-out problem arrived with it.

Failure scenarios

• No result cache, giving quadratic growth nobody notices until the provider's pipeline doubles in duration.
• Verifying against every contract ever published, so a long-dead consumer blocks a release.
• A provider verification that needs real infrastructure, multiplying every cell of the matrix by minutes instead of seconds.
• Verification moved off the gate "temporarily" for pipeline speed, after which contracts stop being enforced and the first break reaches production.
• Fan-out tracked as a count of consumers rather than executions, which hides the publish rate.

Trade-offs

The caching and environment bookkeeping that make fan-out linear are real infrastructure: a broker to operate, version records to keep accurate, a deployment step that queries it. In exchange, verification stays a sub-minute gate as the estate grows.

The alternative - re-verify everything, no bookkeeping - is cheaper to build and correct at small scale. The decision rule is the compute number: below roughly an hour of verification a day keep the simple thing, above it the cache is cheaper than the machines.

When not to use it

Below about ten consumers. The arithmetic is too small to matter, a broker is operational overhead with no payback, and a provider that verifies every contract on every change gets the strongest guarantee available for a few minutes of CI. Build fan-out management at this size and you maintain a scheduler for a problem you do not have.

It is also irrelevant where contracts are not consumer-driven. A public API with unknown external consumers cannot enumerate its contracts, so the governing mechanism is schema compatibility rules and deprecation policy, and there is no matrix to size.

Interview question

Q: "You run a provider with 300 consumers and verification is now the slowest stage in your pipeline. The team wants a nightly batch instead. What do you do, and what number convinces you?"

What a strong answer covers: moving it off the gate removes the guarantee rather than the cost; content-hash caching and verifying only against versions deployed in each environment; the difference between compute hours and publication-to-answer latency; measuring per-run duration before optimising; and the deploy-time compatibility query that makes a nightly batch unnecessary.

Quick check

Quiz: Why is uncached verification fan-out quadratic in consumer count? - Because each consumer's publication triggers a full run against every recorded contract, so executions and contracts per execution both grow with consumer count.

Flashcard: Which single mechanism turns verification fan-out from quadratic to linear? - Caching verification results by (provider version, contract content hash) so unchanged contracts are never re-verified.