Data Governance

Definition

Governance answers four questions for every significant dataset: who owns it, what does it mean, how good is it, and who may see it. Without answers, a data platform accumulates datasets nobody maintains and numbers nobody trusts.

The components that earn their keep

• Ownership. A named team accountable for a dataset's correctness and availability, not a distribution list. An unowned dataset degrades silently.
• A catalogue with definitions. Not a list of tables — a statement of what each field means and how it is derived. Two teams computing "active user" differently is a governance failure, not an analytics disagreement.
• Lineage. Where a number came from and what depends on it. This is what makes both incident response ("which dashboards are affected by this bad load?") and change management ("what breaks if I drop this column?") tractable.
• Quality expectations as tests. Row-count bounds, null rates, referential checks, distribution drift — executed in the pipeline and failing it, so bad data does not reach a dashboard.
• Classification and access control. What is personal, what is confidential, and who may see it, enforced by policy rather than by convention.

Industry example

Operating a large multi-tenant enterprise platform — the position Microsoft occupies with cloud services — makes governance a product requirement rather than an internal hygiene matter. Customers demand to know where their data resides, who can access it, how long it is retained, and how that is proven to an auditor.

The architectural consequence is that governance metadata must be enforced at runtime, not documented. Residency is a routing and placement rule the system cannot violate. Retention is a deletion job that runs. Access is a policy evaluated on every request. A governance model that lives in a spreadsheet describes intentions; one that lives in the request path describes behaviour, and only the second survives an audit.

Failure scenarios

• A catalogue that is not maintained, so it lists tables deleted a year ago and omits the ones people use, and everyone reverts to asking in chat.
• Governance as a review board that slows delivery without improving quality, which produces workarounds rather than compliance.
• Ownership assigned to a platform team for datasets whose semantics only the producing team understands.
• Access controls at the table level when the sensitivity is at the column or row level, so people are either over-granted or blocked from work they need to do.
• No deletion mechanism, so a retention policy exists on paper and nothing has ever been deleted.

Trade-offs

Bought: trust, auditability, faster incident response and safer change. Sold: friction on every new dataset and a standing investment in tooling and stewardship. The failure mode at both extremes is real: no governance produces an untrusted swamp, and heavy governance produces shadow pipelines built to avoid it.

Interview question

"Two dashboards report different revenue for the same month. Walk me through how you find out why, and what governance mechanism would have prevented it."