Swapping the system behind the name: ten years of LinkedIn

LinkedIn wrote Kafka, published it, and in 2025 announced its replacement. Over the same decade it also left its own RPC framework, its own service discovery format, its own key-value store and its single stream-processing engine, and abandoned the one migration it tried without a compatibility layer. This guide reconstructs the five-part shape of that layer from the public record, gives the condition that flips each design choice inside it, and shows from four published incidents how transitions between two versions fail. After reading it you can tell whether a migration you are planning has a seam or a cliff, and you will know the one artefact in the pattern that almost nobody builds.

What you get out of it

• Every completed replacement in LinkedIn's decade went through an indirection layer that implemented the incumbent's interface exactly; the one attempt without one, the lift and shift to Azure, is the one that stopped.
• A private extension to someone else's version number is a dated liability: 3.0-li inserted a field into LeaderAndIsr v3, Apache later gave that version a different meaning, and the bill arrived in 2026 as an off-by-default bridge mode and a two-branch pull request programme.
• The compatibility gate's three properties are all load-bearing: cluster-scoped, dynamically changeable, and off by default, so an instance told nothing behaves like the old fleet and reversing needs no restart.
• The part nobody builds is a sentence: a written exit criterion naming the observable condition under which the compatibility code is deleted. LinkedIn wrote one into a 2023 pull request description, and no postmortem in this corpus describes a seam that was actually removed.
• Shadow reads are how you buy the mismatch taxonomy cheaply: LinkedIn's discovery migration found reconnection storms, data inconsistencies, subscription bugs and TLS misconfigurations while ZooKeeper was still authoritative.

Scope

Why this, now. Apache Kafka 4.0 closed the ZooKeeper path in March 2025 and LinkedIn's public fork is still on a 3.0 branch, so the cost of a decade of private divergence is visible in its repository right now, in September 2026 pull requests.

What it does not cover. It does not cover Northguard's internals beyond what the announcement describes, does not compare Northguard to Kafka as products, does not touch the economics of owning data centres, and says nothing about migrating off a vendor you do not control.