A platform adopts a transformation framework and analysts build hundreds of models. What governance is needed to prevent a tangle?

Why a tangle forms

The framework lowers the cost of creating a derived dataset to almost nothing, which is its value and its risk. Analysts create models freely, each depending on others, and within a year the dependency graph is hundreds of nodes deep with no layering, duplicated logic and no clear source of truth for any metric.

The symptom is several models producing different numbers for the same business concept, and nobody able to say which is right.

The governance that prevents it

1. Layering with enforced dependency direction. Staging models reference sources; intermediate models reference staging; marts reference intermediate. A mart referencing a source directly, or a staging model referencing a mart, should fail the build.

2. One definition per business metric, in a designated place, referenced everywhere. Most disagreement about numbers is vocabulary disagreement in technical costume, and this is the control that removes it.

3. Tests as a requirement, not an option. Uniqueness, non-null, referential integrity, accepted values — declared alongside each model and run on every build. Cheap to write and they catch the failures that otherwise surface as a wrong dashboard.

4. Ownership per model, so unowned models are visible and removable.

5. Documentation as part of the model, since a dataset nobody can interpret is not usable regardless of its correctness.

6. Deletion as routine. Models nobody queries should be removed. Without this, the graph only grows and every change becomes riskier.

The cost control that must accompany it

Materialisation choices per model — view, table or incremental — decided deliberately. A framework that materialises everything as tables rebuilds the world nightly; one that materialises everything as views pushes enormous cost to query time.

Incremental models for large tables, with a defined strategy for late-arriving data, are usually where most of the cost saving is.

The measure

Time from a source change to correct downstream outputs, and the number of models nobody has queried in ninety days. The second is the honest measure of accumulated tangle.