An organisation is migrating from a legacy data warehouse to a modern platform. How should it be sequenced, and why do these migrations so often end with both systems running permanently?

Why they stall

The migration's value is realised only at decommissioning, and decommissioning is the part nobody funds. The new platform is built, the important pipelines are moved, the remaining long tail is unowned or belongs to someone with other priorities — and the organisation now pays for two platforms, maintains two sets of logic, and makes every change twice.

This is worse than either end state, and it is the most common outcome.

The contributing causes are consistent: an unknown consumer nobody can identify, reports whose owners have left, logic embedded in the legacy platform that nobody can read, and a migration scoped without an end date.

The sequence

1. Inventory what is actually used, from query logs and access records rather than from documentation. A large fraction of tables and reports in a mature warehouse have no consumers, and the owners do not know. This step alone frequently removes a third of the scope.

2. Classify by value and health, and decommission the low-value unhealthy quadrant immediately — before migrating anything. Migrating something you should have deleted is the most expensive mistake in the programme.

3. Move ingestion first, not consumption. Land the raw data in the new platform in parallel with the existing pipelines, so the new platform accumulates history while nothing depends on it.

4. Migrate by domain, not by table. A domain is a coherent unit with an owner and a set of consumers; table-by-table migration produces a long period where a report reads from both platforms.

5. Parallel run with reconciliation. Both platforms produce the same outputs; compare them. Run for a full business cycle at minimum, since the rare monthly and quarterly calculations are where the differences concentrate and a two-week comparison systematically misses them.

6. Triage every discrepancy into one of four categories — new system wrong, legacy system wrong (more common than expected, and the hardest case because correcting it changes historical figures), specification ambiguous, or inputs differed, which is frequently the largest early category. No discrepancy is closed without a category and an explanation, or a genuine defect is closed as "rounding".

7. Switch consumers, with the legacy still running.

8. Stop the legacy pipelines, then archive the data and decommission — with a date, in the same funded project.

What must be arranged at the start

• Decommissioning funded and dated in the original scope, or it does not happen.
• A freeze on new development in the legacy platform, or the target moves.
• The long tail owned explicitly — the last 10% are unowned reports and departed owners, and someone must be budgeted to do that work for them, because they will not do it themselves.
• Archive-and-shutdown as a standard path, since the requirement is usually to retain the data, not to run the platform — exporting to open-format archival storage satisfies retention and unlocks a large fraction of the estate.