intermediate 3 min answer Multiple choice

A BI team replaces a 9-table star schema with a single denormalised table of 240 columns because dashboards were slow and analysts kept joining at the wrong grain. Six months later the table is 4 TB across roughly 3 billion rows, the nightly build takes 5 hours, and correcting one product's category requires a full rebuild. Which property did the flattening actually trade away?

one big tabledenormalisationscdrebuild costgrain
Pick one
Show the full answer Hide the answer

What the flattening bought

Two real things. The BI tool can no longer join at the wrong grain, because there is nothing to join, which removes the most common source of wrong numbers on a dashboard. And in a columnar store a 240-column table is not expensive to read: a query touching 4 columns reads 4 columns, so the intuition that wide tables are slow to scan is simply false here.

What it gave up and when the bill arrives

Normalisation is not a storage optimisation, it is a statement about the unit of change. In the star schema, a product's category lived in one row of dim_product. After flattening it lives on every fact row that mentions that product, so a correction is a rewrite of the affected partitions of a 4 TB table.

The bill arrived on the day someone asked for a correction, which is typically three to six months in, well after the decision has been praised for the dashboard speedup. Two second-order effects land with it:

  • The table has become an accidental Type 2 dimension with no effective dates. Attributes are frozen as they were at load time, which is often the desired behaviour, but nothing records which version was in force, so a restatement cannot be reasoned about.
  • Incremental build is gone for any change to a dimension attribute. New facts append cheaply; an attribute correction touches history, so the nightly job becomes a full rebuild and the 5 hours is structural rather than a tuning problem.

What to change now

Keep the wide table, demote it. Build it from the star schema as a materialisation, not as the source of truth. Then move the roughly 10 volatile attributes out of the wide table and leave them as a join to a small dimension, so corrections touch one small table and the rebuild becomes a partition-scoped merge. Flattening stable attributes is free; flattening volatile ones is the whole cost.

Why the other options fail

  • "Nothing was lost and the build needs more compute." More compute shortens a 5-hour rebuild to a 3-hour rebuild and leaves the correction path untouched. This is the answer that keeps a team buying capacity for two years.
  • "Column pruning which wide tables defeat." Columnar formats prune by column by design, and 240 columns cost nothing to a query reading 4. This is the plausible-sounding physics answer and it is backwards.
  • "Referential integrity enforced by the warehouse." Most analytical engines do not enforce foreign keys at all, so there was nothing to lose. The integrity in a star schema comes from the load process, which the team still owns.
  • "Query concurrency because wide tables cannot be cached." Result caches key on the query, not on the table's width. Concurrency did not change, and dashboards got faster, which is why the decision looked good for six months.

When this is the wrong answer

When the attributes are immutable by nature, the criticism does not apply. Event properties captured at the moment of the event never need correcting, which is why event tables are legitimately wide and entity tables are not. If nothing in the 240 columns can ever be restated, flattening has no bill to arrive and the team should be left alone.