Metric Certification

The recurring tension in analytics is that teams want to explore freely and finance needs numbers that reconcile. Treating this as one dial — more governed or more open — produces a setting that slows analysts and still leaves the numbers disagreeing.

Certification resolves it by moving the control to a different place: not on who may query, but on what may be asserted.

The tiers

• Certified. A deliberately small set with owners, contracts, tests, lineage and change control. Slow to change by design. The only permitted source for a board pack, a regulatory filing or an external claim.
• Curated. Modelled, documented, reliable, lightly governed. What teams build on day to day.
• Raw and exploratory. Open, no guarantees, explicitly marked unsuitable for reporting.

Analysts work freely in the lower tiers. The gate applies at the point a number leaves the team as a claim.

Implementation patterns

• Visible tier markers on every output, so a chart pasted into a deck carries where it came from. Without this the tiers exist and nothing enforces them — this is the single most commonly omitted piece.
• Certified metrics defined in the [[semantic-layer]], so the definition and its execution cannot drift.
• A promotion path from exploratory to certified that is genuinely usable, or good work stays in the wrong tier and gets used anyway.
• A certified tier good enough to use. If it lacks a dimension someone needs, they route around it and the governance becomes theatre. This is an engineering obligation, not a policy one.
• A named arbiter for when two numbers disagree, so the dispute resolves instead of recurring quarterly.
• Usage measured on certified assets, so the set can be kept small — certification of everything is certification of nothing.

Industry example

Organisations with thousands of dashboards and several conflicting definitions of the same metric almost always have the same root cause: metric logic lives in dashboards rather than in a shared layer. An analyst could not find a trustworthy existing dashboard, built one encoding their own interpretation, used it once and never deleted it — and the next analyst repeated the cycle because discovery is harder than rebuilding.

The interventions that work start with the cheapest and least contested: turn off scheduled refresh for dashboards nobody opens, then certify a small set built on shared definitions and make those the discoverable default. Restricting dashboard creation is the intervention to avoid — it moves the work into spreadsheets, where it is invisible and worse.

Failure scenarios

• Certifying too much, which dilutes the signal and makes the tier unmaintainable.
• No tier marker on outputs, so consumers cannot tell certified from exploratory.
• A certified tier that is incomplete or slow, guaranteeing bypass.
• No arbiter, so definitional disputes recur indefinitely.
• Certification as a one-time event, after which the definition drifts and the badge misleads.

Trade-offs

Certification is real ongoing cost: owners, tests, change control and a review cadence. That cost is why the set must stay small, and keeping it small means refusing certification for things people want certified.

The compensating benefit is that trust becomes legible. A number that carries a certification marker can be used without re-derivation, and the time analysts currently spend reconciling conflicting figures is usually larger than the cost of maintaining the tier.

Interview question

"Two teams present different revenue numbers in the same meeting and both can defend their query. What should have existed to prevent it — and what should exist to resolve it in the room?"