Learning paths
Original
LinkedIn, behind the front page
A feed in 300 milliseconds for 1.2 billion members: the whole design, its decisions and its traps, told in a newspaper printing house.
Drag to look around · arrow keys step through · T for the text view
A narrated 3D walkthrough of the LinkedIn network design: records kept apart from rebuildable copies, one feed page followed through its 300 millisecond budget, push, pull or both, privacy on every read, safe retries and what each failure does.
Walk an engineer or architect through the LinkedIn Professional Network use case: how it works, why it is built this way and what would break it, at the level of the diagram views and decision records, with the stack, the patterns and a hypothetical product interface on screen.
What is real, and what is a scenario
- The requirement: availability, latency, consistency and the feed, privacy and reliability design (ask.md)
- Published scale: 1.2 billion members, a 270-billion-edge graph at 2 million queries a second, 7 trillion Kafka messages a day (README.md, view 01)
- The component stack, the 33 decision records and the alternatives that lost (docs/adr.md)
- The latency budget split of 300 milliseconds, the recovery numbers and the degradation contract (views 14, 22, 26)
- The facts cited about LinkedIn's own systems come from the use case's decision record and are not re-fetched in this run
- The product screens are an illustrative interface, not the real product, and carry no brand
- The stage-by-stage latency split of the 300 millisecond feed budget is the use case's own assumption (view 14)
- The 250,000 feed requests a second planning peak, and the 10,000-follower fan-out threshold, are the use case's assumptions, not LinkedIn figures
- The three traffic levels, how latency grows with load (peak at 55 percent utilisation), and the 25 percent deadline slack are illustrative models
- A reader following 400 accounts, 20 of them large, is an illustrative reader for the fan-out calculator
- One colo holding a quarter of members assumes members are pinned evenly across four colos
- The metaphor (wire room, vaults, pigeonholes) is a teaching device, and the blueprint view maps each part to its real component
- Meters are illustrative, and the advice to start with one database and a log is this walkthrough's own
What you leave with
- Why the records and the copies are kept apart, and how copies are rebuilt
- How a 300 millisecond feed page is budgeted, stage by stage
- Why hybrid fan-out beats pure push or pure pull, and what it costs
- Where privacy, idempotency and failure contracts hide, and what to ask of your own design
Sources
- Apache Kafka at 7 trillion messages a day (LinkedIn, 2019) 7 trillion messages a day across 100+ clusters (as cited in the use case's decision record; not re-fetched in this run)
- FollowFeed (LinkedIn, 2016) Pull-based feed index, 720 partitions, first-pass ranking on the index nodes (as cited in the use case's decision record; not re-fetched in this run)
- TrafficShift: load testing at scale (LinkedIn, 2017) Four data centres serving traffic, drained for load tests and incidents (as cited in the use case's decision record; not re-fetched in this run)
- Open-sourcing Brooklin (LinkedIn, 2019) Change capture and mirroring; replaced MirrorMaker clusters (as cited in the use case's decision record; not re-fetched in this run)
- Open-sourcing Databus (LinkedIn, 2013) First change-capture system (as cited in the use case's decision record; not re-fetched in this run)
- Hodor (LinkedIn, 2022) Overload detection and remediation (as cited in the use case's decision record; not re-fetched in this run)
- Open-sourcing Venice (LinkedIn, 2022) Derived-data platform; 200+ hybrid datasets (as cited in the use case's decision record; not re-fetched in this run)
- Instant messaging at LinkedIn (LinkedIn, 2016) Server-Sent Events messaging on Play and Akka (as cited in the use case's decision record; not re-fetched in this run)
3D with three.js (MIT) · Type: Fraunces and Spline Sans Mono (SIL Open Font Licence), Inter · Narration voice: Kokoro-82M v1.0, voice af_heart (kokoro-onnx 0.6.1). Apache-2.0 (Kokoro-82M weights).
More walkthroughs
Learning paths
Original
Learning paths
Original
Viveka on any position
For anyone from a first-month beginner to a titled player: train the Viveka method on a flat board you style yourself, on five lessons or any of 5,494 held-out positions drawn at random, narrated as it computes.
Learning paths
Original