Four business units each run their own metadata catalogue and data lake after a decade of acquisitions. Analysts in every unit now need to join tables owned by the others. The approved programme is a single consolidated catalogue in 18 months. There is live analytics traffic, no cutover window, and the cross-unit reporting requirement lands this quarter. Sequence a migration that makes cross-unit joins work in one quarter rather than in eighteen months.

The sequence

The move that collapses eighteen months into one quarter is to separate access from ownership. The programme conflated them: it assumed that querying across four catalogues requires one catalogue.

1. Put a routing proxy in front of the four catalogues. Clients connect to one endpoint; the proxy resolves each database or table reference to the catalogue that owns it and forwards the metadata request. Nothing moves, nothing is rewritten, and every existing client keeps working against its own catalogue directly. This is the step that delivers the quarter's requirement. Reversible: switch the client's endpoint back.
2. Resolve the name collisions before anyone depends on the proxy. Four independently grown catalogues will have several customer, bookings and default databases. Federation gives you one namespace, so collisions become query-time ambiguity. Publish a prefixing convention, apply it in the proxy's mapping first, then in the source catalogues over the following quarter.
3. Point one unit's analysts at the proxy, read-only, with their own catalogue still reachable. Run both for two weeks and compare row counts and schemas on a sample of cross-unit queries.
4. Move writes last, and only where ownership is genuinely changing. Federation does not need write consolidation to deliver reads.
5. Decide whether consolidation is still worth doing, with the pressure off. Often the answer is partial: consolidate two units that already share a domain, federate the rest permanently.

Where data can diverge, and how you would know

The divergence is not in the data - it is in the metadata. A partition added directly in one unit's catalogue appears instantly through the proxy; a schema evolved in a way the proxy's version cannot express does not. Instrument the proxy for per-catalogue error rates by operation, and run a nightly reconciliation that lists tables visible in a source catalogue but not through the proxy. A table that silently stops being visible is the failure mode that produces a wrong report rather than an error.

The point of no return

It arrives at step 4, not step 1. Until writes move, every step is a routing change. Once a unit's tables are created through the proxy by default, the proxy is a production dependency on the critical path of every query in four business units - and its configuration, typically loaded at startup, becomes a change-managed artefact. Plan its availability and its rollout like a database, because that is what it now is.

The grounding, and how long it really takes

Expedia Group open-sourced Waggle Dance in 2017 (Apache-2.0) as exactly this: a Hive federation service that routes metastore requests so that datasets across several independent Hive deployments can be queried concurrently. Its documented motivation was the dataset silos that appeared as the organisation moved from on-premises clusters to the cloud, and the group packaged the surrounding components - metastore, federation, cleanup - for deployment as Apiary. Apache Hive's own design documentation notes the two real costs: federation is configured externally, and independent metastores collide on names.

Realistic timing: proxy in front of four catalogues and read-only for one unit, four to eight weeks. Name collisions resolved across four estates, one to two quarters, because it needs every owner's agreement. Consolidation, if you still want it, as long as the original plan said - the difference is that the business is no longer waiting for it.

When this is the wrong answer

When the four catalogues are small and already converging. If two of the four hold a dozen tables each and their owners are being merged anyway, a federation proxy adds a permanent production dependency to avoid two weeks of work. Federate when ownership is genuinely staying separate.

And where copying Expedia's choice would be a mistake: it fits an estate whose business units will keep owning their own data platforms. A single-tenant organisation that federates is building a routing layer to avoid a conversation, and will end up operating both the proxy and the consolidation. Federation also does not reconcile semantics - four definitions of a booking stay four definitions, and the proxy makes them look like one namespace, which is a new way to produce a confidently wrong join.