Governance Artifact design intermediate

Data Contract

A producer's binding promise about a dataset — schema, semantics, freshness, quality and how it will change — enforced in the pipeline rather than in a meeting.

Data Platform Build Integration Programme Cloud-Agnostic Cross-Industry Engineering Architecture Review Board

The interface definition for data. What makes it a contract rather than documentation is that a violation fails a build: the producer's CI checks the schema and semantic rules before the change ships, and the platform checks freshness and quality continuously after it does.

The shape

dataset: fulfilment.order_events
version: 2.3.0
owner: fulfilment-engineering        # team, plus a named person on call
consumers: [sales.customer-360, finance.revenue-recognised, ml.delivery-eta]

schema:
  order_id:        {type: uuid,      required: true,  pii: false, unique: true}
  customer_id:     {type: uuid,      required: true,  pii: true,  classification: confidential}
  status:          {type: enum,      required: true,  values: [PLACED, CONFIRMED, PICKING, SHIPPED, DELIVERED, CANCELLED]}
  total_minor:     {type: integer,   required: true,  unit: minor_currency_units}
  currency:        {type: string,    required: true,  format: ISO-4217}
  occurred_at:     {type: timestamp, required: true,  timezone: UTC}

semantics:
  grain: one row per order status transition
  occurred_at: when the transition happened, NOT when it was ingested
  total_minor: excludes tax and shipping
  late_arrival: events may arrive up to 2h late; use occurred_at for windowing

guarantees:
  freshness:    p99 end-to-end < 5 min
  completeness: >= 99.9% of transitions within 24h
  uniqueness:   (order_id, status, occurred_at) unique
  availability: 99.5% monthly

change_policy:
  additive:  minor version, no notice required
  breaking:  major version, 90 days notice, both versions served during overlap
  removal:   180 days notice, consumer sign-off required

enforcement:
  producer_ci:  schema + enum + uniqueness checks block the merge
  platform:     freshness and completeness monitors, alert to owner
  on_breach:    consumers notified; producer owns remediation

When you produce it

For any dataset another team builds on. The trigger is usually the third time a producer's unannounced schema change broke a downstream pipeline overnight.

Who reads it

Producer teams, who are agreeing to it. Consumer teams, who are relying on it. The platform, which enforces it automatically.

What good looks like

Semantics, not just schema. Most breakages are meaning changes that pass every type check — a timestamp that quietly becomes ingestion time, an amount that starts including tax.
Freshness and completeness are numbers with monitors behind them.
The change policy distinguishes additive from breaking, with real notice periods and an overlap window.
Enforcement is named per rule, and at least the schema rules run in the producer's CI. A contract checked only downstream tells you after the fact.
PII and classification per field, so downstream controls can be automatic.

Common mistakes

Schema only, which catches the minority of breakages.
No enforcement, making it a wiki page with a version number.
No named owner, so a breach has no addressee.
Contracts on every table. Apply them to published data products; internal intermediates do not need one and the overhead will kill adoption.