A data platform introduces data contracts and pipelines still break when producers change schemas. What was missing?

What was missing

Enforcement at the producer. A contract that consumers declare and producers do not check is documentation: the producer changes the schema, the contract is violated, and the violation is discovered downstream by a broken pipeline.

A data contract is only a contract if the producer's build fails when it is broken.

What a working contract requires

• Enforcement in the producer's pipeline, so a breaking change is blocked before it ships rather than detected after.
• A schema with compatibility rules: additive changes allowed, removals and type changes requiring a version.
• Semantics documented alongside the schema — what each field means, its units, when it is populated, what it does not mean. The negative statements are the valuable ones, and a semantic change with an identical shape passes every check while consumer behaviour quietly diverges.
• Quality expectations, not just structure: null rates, value ranges, uniqueness, freshness. A field that is present and always null satisfies a structural contract and breaks its consumers.
• A named owner on the producing side, with an SLO the consumers can rely on.
• Consumer registration, so a producer can assess a change — without it, a producer cannot change anything safely and therefore does not, which is the coupling the contract was meant to remove.

Why it is organisational rather than technical

The mechanism is straightforward. The difficulty is that it makes the producing team accountable for something they previously were not, and a producing team that considers the data an internal implementation detail will resist it.

That is the real work, and it requires the platform to make compliance cheap — generating the contract from the schema, providing the tests, and absorbing the migration cost when a contract must change.

Where contracts genuinely do not fit

Exploratory and one-off data. Contracting everything makes the platform rigid and the process bypassed.

Contract the interfaces that other teams depend on, and leave the rest ungoverned — the same distinction as between a public API and an internal function.