advanced 3 min answer

A provider service has 40 consumers. Each consumer publishes its contract on every merge and merges about 6 times a day. A full provider verification run takes 90 seconds. Roughly how much verification work per day - and what does the number rule in or out?

contract testingestimationci costcachingpact
Show the full answer Hide the answer

The assumptions, stated

  • 40 consumers × 6 merges a day = 240 contract publications a day.
  • The provider itself merges 8 times a day, each merge verifying against all recorded contracts.
  • A verification run replays all 40 contracts in 90 seconds, so roughly 2.2 seconds per contract interaction set.
  • Working hours concentrate the publications: about 70% land in six hours.

The arithmetic

With result caching, keyed on the pair (provider version, contract content hash), the work is linear. Each publication needs only its own contract verified against the current provider version: 240 × 2.2 s ≈ 9 minutes of compute a day, plus 8 provider-side full runs at 90 s ≈ 12 minutes. Total under half an hour of CI time. Trivial.

Without caching, every publication triggers a full verification of all 40 contracts: 240 × 90 s = 21,600 s ≈ 6 hours of CI a day, and it is quadratic in consumer count. At 400 consumers with the same publish rate the same model gives 2,400 publications × 400 contracts × 2.2 s ≈ 585 hours of compute a day - 24 machines running continuously to re-verify contracts that did not change.

The second number is the one that matters, because most publications are byte-identical to the previous one. A consumer merging a CSS change republishes the same contract. Content-hashing the contract and skipping unchanged pairs removes 90% or more of the work, which is why the caching assumption dominates the estimate far more than the publish rate does.

Which assumption dominates the error

Not the 90 seconds, and not the merge rate. It is whether the broker can answer "has this exact pair already been verified". Get that wrong and the estimate is off by two orders of magnitude. The second-order term is contract granularity: a provider with one 400-interaction contract per consumer pays very differently from one with 12 small ones, because the per-run fixed cost (container start, database seed) is usually larger than the per-interaction cost.

What the number rules in or out

At 40 consumers, nothing is ruled out - even the uncached version fits in 6 hours of CI, which costs less than a day of an engineer's time per month. So do not spend the quarter building a verification scheduler.

What breaks first at this scale is not compute but latency on the deploy path: a consumer that must wait for the provider's verification queue before it can deploy has acquired a cross-team synchronous dependency, which is the thing contract testing was adopted to remove. The number to watch is time from contract publication to a usable compatibility answer; when that exceeds the consumer's own pipeline duration, add caching and parallel verification, in that order.

When this is the wrong answer

When the provider's verification needs real infrastructure. If a run requires a seeded database and a message broker and takes 11 minutes rather than 90 seconds, the arithmetic changes sign: 240 publications × 11 minutes is 44 hours a day and caching is no longer an optimisation but the only way the practice functions. Measure the run before accepting any of these numbers, and if the run is slow, fix the run - a verification that cannot be executed on demand is documentation with a build badge.