Three teams report different numbers for the same metric and each is confident. What is the architectural cause, and what fixes it?
Show the full answer Hide the answer
The cause
The metric's definition lives in three places — three dashboards, three queries, three sets of filters — and each encodes slightly different assumptions: whether cancelled orders count, whether the timezone is the customer's or the platform's, whether test accounts are excluded, whether an order with a partial refund counts once or partially.
Every version is defensible and none is authoritative, so the argument cannot be resolved by inspecting the numbers.
What fixes it
A semantic layer: the metric defined once, in one place, and consumed by every tool. Dashboards, ad-hoc queries and downstream systems reference the definition rather than reimplementing it.
The properties that matter:
- One definition, version-controlled and reviewed like code, so a change is deliberate and traceable.
- Consumed rather than copied. A layer that produces documentation for humans to reimplement solves nothing.
- Dimensions defined alongside metrics, since most disagreements are about the slice rather than the measure.
- A named owner per metric, because a definition with no owner is one nobody can change or defend.
What the semantic layer does not fix
- Upstream data quality. A perfectly defined metric over incorrect data is precisely wrong.
- Disagreements that are genuinely about the business question. If finance and product mean different things by "active customer", the answer is two metrics with distinct names — not one definition imposed on both, which produces a number neither trusts.
- Adoption. A semantic layer nobody uses because it is slower or less flexible than writing SQL directly has changed nothing.
The governance decision underneath
Which metrics are governed and which are not. Attempting to govern every number produces a bottleneck and is abandoned; governing the twenty that appear in board reporting, regulatory filings and team objectives is achievable and captures most of the value.
Everything else remains self-service and ungoverned, explicitly — and stating that boundary is what prevents governance from becoming an obstacle to analysis.