advanced 3 min answer

A customer dimension has always been Type 1 - attributes overwritten in place. Compliance now requires every order to report the segment and region the customer had at the time of the order, three years back. 140 reports read the dimension and 2.1 billion fact rows join it on the natural customer key. Sequence the conversion under live reporting.

scdtype-2migrationsurrogate-keysrestatement
Show the full answer Hide the answer

The sequence

  1. Decide per attribute, not per table. Of 34 attributes, perhaps 6 need history. Versioning all 34 turns every address correction into a new row and multiplies the dimension by the noisiest field in it. Reversible: this step is a document.
  2. Find out how much history actually exists. Candidates: raw landed snapshots in bronze, the change log if its retention reaches back, a source-side audit table, or nothing. Where history cannot be recovered, history begins at cutover - and that date must appear on the reports, not in a wiki.
  3. Build the versioned dimension beside the old one. Surrogate key, valid_from, valid_to with a high sentinel rather than NULL, is_current, natural key retained. Seed one version per customer from current state with valid_from set to the declared history-start date, then apply whatever history step 2 recovered. Reversible: nothing reads it yet.
  4. Publish the old shape as a view over the new table filtered to is_current. All 140 reports keep working unchanged. This is the step that buys the time for everything after it.
  5. Change the fact load to resolve the dimension version at load time by the order's date, writing the surrogate key onto new fact rows. Backfill the surrogate key onto the 2.1 billion existing rows in date-partitioned batches, checking row counts per partition as you go - a mismatch here is a silent multiplication later.
  6. Point of no return: switching a report to join on the surrogate key. That is when historical numbers change. Do it report by report, with a parallel run for anything finance signs.
  7. Retire the natural-key join last, and keep the column.

Where data can diverge, and how you would know

Revenue by region will move for exactly the customers who changed region. Reconcile the old and new totals per period and classify every difference before anyone sees a dashboard: a customer who moved from EMEA to APAC in March is legitimate divergence, a figure that moves by 4% with no corresponding movers is fact grain multiplication from a join missing its effective-date predicate. Facts older than the history-start date bind to the earliest version, so the oldest periods will match the old report exactly - that is a feature, and it should be stated.

Rollback

Steps 1-5 are reversible: drop the new table, revert the loader. From step 6, rollback means repointing a report to the is_current view, which restores the old numbers but not the signatures already given on new ones. Keep the view alive for a release after the last report moves.

How long it really takes

Weeks to months, and the SQL is not the long pole. Step 1 waits on a business decision and step 6 waits on 140 report owners. A team that budgets for the backfill and not for the reconciliation meeting will be late.

When this is the wrong approach

If nobody needs as-at reporting and the real requirement is an audit trail, choose the Type 1 dimension plus an append-only change-log table beside it. It answers "what changed and when" because every write is recorded with its timestamp, and it does so at a fraction of the cost: no surrogate keys, no backfill of 2.1 billion fact rows, no 140 report conversions. The retrofit above is justified only when a report has to show the world as it was, which since the 2013 edition of the Kimball method has been the textbook trigger for Type 2 - and even then only for the handful of attributes that reporting slices by.

The failure to avoid is the half-finished conversion: a versioned dimension in place while some reports join on the natural key and others on the surrogate. The two sets then disagree by exactly the customers who moved, nobody can say which is right, and the usual response is to patch one report rather than finish the migration.