Evidence ledger 22 sources Checked 15 Sep 2026

Evidence ledger

One row per claim in Own the layer above the engine: ten years of LinkedIn's data platform: who published it, what grade it carries, when it was written, when the link was last checked, and the quote or figure it rests on. Nothing in the guide is cited from memory, so anything not in this table is not in the guide.

Field guide: Own the layer above the engine: ten years of LinkedIn's data platform, read from its own repositories Category: data-platforms · Research date: 2026-09-15 · All links fetched 2026-09-15.

Corpus limit, stated up front. This session's network egress policy reached github.com, raw.githubusercontent.com, repo1.maven.org, pypi.org, registry.npmjs.org, hub.docker.com and proxy.golang.org. It did not reach engineering.linkedin.com, venicedb.org, openhousedb.org, linkedin.jfrog.io, docs.pinot.apache.org, arxiv.org, www.usenix.org, medium.com or any video host. So this corpus contains no conference talks, no peer-reviewed papers and no company engineering-blog posts. Every claim below comes from a repository file, a package registry, a container registry or a security advisory. Where LinkedIn's own blog is the only place a motivation was stated, the guide says so and stops rather than reconstructing the sentence from memory.

# Org Title Tier Published Checked URL Claim taken from it Supporting quote or figure
1 LinkedIn rest.li README, deprecation banner adr 2026 (archived 2026-07-22) 2026-09-15 https://raw.githubusercontent.com/linkedin/rest.li/master/README.md The framework that carried LinkedIn's service layer was deprecated in favour of gRPC, with no drop-in path "This project is deprecated. rest.li is no longer actively maintained and the repository will be archived (read-only) on July 22, 2026. LinkedIn has moved to gRPC internally; gRPC is not a drop-in replacement."
2 LinkedIn rest.li README, stated reason adr 2026 2026-09-15 https://raw.githubusercontent.com/linkedin/rest.li/master/README.md The reason given is capability and community, not cost "gRPC will offer better performance, support for more programming languages, streaming, and a robust open source community."
3 LinkedIn rest.li issue #1178 adr opened 2026-06-30 2026-09-15 https://github.com/linkedin/rest.li/issues/1178 External adopters were told to migrate themselves; the artifacts stay, the maintenance does not "After July 22, 2026, the repository will be read-only. We will not accept new issues or pull requests"; "We are not providing a drop-in replacement."
4 LinkedIn rest.li CHANGELOG, 29.85.15 source 2026-06-29 2026-09-15 https://raw.githubusercontent.com/linkedin/rest.li/master/CHANGELOG.md The deprecated framework's final release, three weeks before the archive date, is xDS work "## [29.85.15] - 2026-06-29 - Add D2ClientConfig.subscribeToIndisObserverCluster (default false): when enabled, the xDS-based D2 client subscribes to the IndisRegistryObserver D2 service, cluster, and URIs so it receives and caches the live observer endpoint set over xDS."
5 LinkedIn rest.li CHANGELOG, 2026-04-30 entry source 2026-04-30 2026-09-15 https://raw.githubusercontent.com/linkedin/rest.li/master/CHANGELOG.md The last year of work on the old framework is instrumentation of the new protocol "Added recordRequestSizeBytes metric to XdsClientOtelMetricsProvider to track the serialized byte size of outgoing XDS discovery requests, enabling detection of clients approaching gRPC's per-message size limit."
6 LinkedIn rest.li gradle.properties source master branch, 2026 2026-09-15 https://raw.githubusercontent.com/linkedin/rest.li/master/gradle.properties The repository's own version is 29.85.15 under the pegasus group version=29.85.15; group=com.linkedin.pegasus
7 Maven Central com.linkedin.pegasus:restli-server version listing source 2013-02-15 to 2019-12-17 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/pegasus/restli-server/ Public publication stopped at 27.7.18 in December 2019, seven years before the archive notice 34 versions, oldest 1.8.3 2013-02-15, newest 27.7.18 2019-12-17
8 Maven Central com.linkedin.pegasus:restli-client version listing source 2013-02-15 to 2019-12-17 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/pegasus/restli-client/ The client artifact stops on the same day, so this is a publishing decision and not one stale module 34 versions, newest 27.7.18 2019-12-17
9 LinkedIn Voldemort README archive notice source notice current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/voldemort/voldemort/master/README.md The Dynamo-style store was retired in 2018 and its use cases moved to Venice "Voldemort is no longer under development. LinkedIn was the primary maintainer and user of Voldemort, and stopped all production usage in 2018. Most of the Voldemort Read-Only use cases and some of the Read-Write use cases have migrated to Venice, which is actively maintained and also open sourced."
10 LinkedIn Venice README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/master/docs/README.md Venice is defined as a derived data platform with five stated characteristics, and it is not published to Maven Central "Venice is a derived data storage platform"; "High throughput asynchronous ingestion from batch and streaming sources"; "Active-active replication between regions with CRDT-based conflict resolution"; "currently, Venice dependencies are not published on Maven Central and therefore require adding an extra repository definition"
11 Maven Central com.linkedin group directory source listing as of 2026-09-15 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/ The public Java estate is 36 artifact directories and contains no venice entry Entries include coral, dagli, feathr, gobblin, iceberg, parseq, pegasus, transport; no venice
12 LinkedIn Venice read APIs documentation source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/user-guide/read-apis/index.md The read path is a portfolio of three clients distinguished by how many network hops remain Thin Client "2 (via Router)" "< 10ms"; Fast Client "1 (direct)" "< 2ms"; Da Vinci Client "0 (local)" "< 1ms"
13 LinkedIn Venice Da Vinci Client documentation source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/user-guide/read-apis/da-vinci-client.md The lowest-latency option is to move the dataset into the application process, and it is priced in disk and bootstrap time "The Da Vinci Client is a stateful, embedded caching client that eagerly loads Venice data into local RocksDB storage."; trade-offs "Higher memory/disk footprint (stores full dataset locally)"; "Higher startup time (must bootstrap full dataset on first launch)"
14 LinkedIn Venice Improvement Proposals index adr current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/contributing/proposals/index.md Venice runs a public proposal process, and four of its six proposals are still under discussion "VIPs document significant design decisions and architectural changes."; VIP-1 Accepted, VIP-2 Under Discussion, VIP-3 Under Discussion, VIP-4 Accepted, VIP-5 Under Discussion, VIP-6 Under Discussion
15 LinkedIn VIP-3: Rust Server Read Path adr status Under Discussion 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/contributing/proposals/vip-3.md The current frontier is removing the last JVM from the serving path, driven by AI workloads "The Fast Client ... helps latency by eliminating the router hop and most importantly by eliminating an entire JVM from the hot path."; "The Venice read path should be completely free of garbage collection. Venice should be able to serve single get read requests, end-to-end (i.e. as measured by the client), at a p99.99 latency under 1 millisecond."; "The demand for AI use cases powered by Venice is growing on a steep trajectory along all dimensions"
16 LinkedIn VIP-4: Store Lifecycle Hooks adr status Accepted 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/contributing/proposals/vip-4.md Region-by-region control of a push is a first-class requirement of a multi-region derived store "This proposal is to add new configs and hooks which can be used both to monitor and control the push job in a finer-grained manner than is possible today. In particular, this proposal focuses on the way that each individual region is handled."
17 LinkedIn Venice write path documentation source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/contributing/architecture/write-path.md Venice is assembled from other people's engines: Kafka topics for versions, ZooKeeper for metadata, Helix for partition assignment "VT
18 LinkedIn Venice pull request #2795, closed unmerged source closed 2026-08-05 2026-09-15 https://github.com/linkedin/venice/pull/2795 A described data-loss race in version swap was closed by automation for inactivity rather than decided Bot comment: "Closing this pull request due to 37 days of inactivity. This is not a judgment on the value of the work. If you would like to continue, please reopen or open a new PR and we will be happy to take another look."
19 LinkedIn Venice closed, unmerged pull requests, sorted by comments source listing as of 2026-09-15 2026-09-15 https://github.com/linkedin/venice/pulls?q=is%3Apr+is%3Aclosed+is%3Aunmerged+sort%3Acomments-desc The heaviest unmerged work sits on the Da Vinci and Fast Client edges of the read path #2795 "[dvc][cc] Fix unsafe DVRT CDC version swap" (91 comments); #2750 "[server][da-vinci] Add config-gated Netty backpressure to blob transfer client" (78); #2819 "[fast-client] External-storage hooks: listeners + deserialization seam" (71)
20 Docker Inc venicedb/venice-router image metadata source registered 2022-09-24, updated 2026-05-27 2026-09-15 https://hub.docker.com/v2/repositories/venicedb/venice-router/ Venice's public distribution dates from its 2022 open sourcing and is modestly pulled date_registered: 2022-09-24, last_updated: 2026-05-27, pull_count: 4320
21 Apache Software Foundation Apache Pinot README casestudy current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/apache/pinot/master/README.md The query-at-read-time half of the portfolio, with LinkedIn's stated production scale "Pinot was originally built at LinkedIn to power rich interactive real-time analytic applications"; "At LinkedIn, Pinot powers 50+ user-facing products, ingesting millions of events per second and serving 100k+ queries per second at millisecond latency."
22 Maven Central org.apache.pinot:pinot-core version listing source 2019-02-15 to 2026-06-04 2026-09-15 https://repo1.maven.org/maven2/org/apache/pinot/pinot-core/ The donated engine is the one still shipping seven years later 24 versions, oldest 0.1.0 2019-02-15, newest 1.5.1 2026-06-04
23 GitHub Advisory Database GHSA-qj9p-jvmw-82rh, CVE-2022-26112 postmortem 2022-09-25 2026-09-15 https://github.com/advisories/GHSA-qj9p-jvmw-82rh The analytics engine shipped script execution enabled by default "Apache Pinot has Groovy Function support enabled by default"; Critical, CVSS 9.8; affected "< 0.11.0", patched "0.11.0"; CWE-94
24 GitHub Advisory Database GHSA-8gj9-r4hv-3jjw, CVE-2024-39676 postmortem 2024-07-24 2026-09-15 https://github.com/advisories/GHSA-8gj9-r4hv-3jjw An unauthenticated controller endpoint disclosed configuration, fixed by adding role-based access control "Apache Pinot: Unauthorized endpoint exposed sensitive information"; High, CVSS 8.7; affected versions "0.1" through before "1.0.0"; the /appconfigs endpoint exposed OS details, heap size and ZooKeeper paths
25 GitHub Advisory Database GHSA-6jwp-4wvj-6597, CVE-2024-56325 postmortem 2025-04-01 2026-09-15 https://github.com/advisories/GHSA-6jwp-4wvj-6597 Authentication could be skipped by shaping the request path, giving account creation to anonymous callers "Apache Pinot Vulnerable to Authentication Bypass"; Critical, CVSS 9.3; "If the path does not contain / and contain., authentication is not required."; affected 0.8.0 to < 1.3.0; CWE-288
26 GitHub Advisory Database GHSA-73cv-556c-w3g6, CVE-2026-49257 postmortem 2026-05-25 2026-09-15 https://github.com/advisories/GHSA-73cv-556c-w3g6 The same trusted-network default reappeared in the AI tool layer in front of the same engine "mcp-pinot: Unauthenticated tool invocation via default oauth_enabled=False + host 0.0.0.0 bind"; Critical, CVSS 10.0; affected <= 3.0.1, patched 3.1.0; any network-adjacent caller can invoke every tool with the server's Pinot credentials
27 GitHub Advisory Database GHSA-r8gm-v65f-c973, CVE-2022-39366 postmortem 2022-10-28 2026-09-15 https://github.com/advisories/GHSA-r8gm-v65f-c973 The metadata control plane accepted unsigned tokens, so anyone could be anyone "acryl-datahub missing JWT signature check"; Critical, CVSS 9.9; StatelessTokenService used only the parse method without signature verification; affected "< 0.8.45", patched "0.8.45"; CWE-347
28 Docker Inc linkedin/datahub-gms image metadata source registered 2020-05-29, last updated 2024-04-02 2026-09-15 https://hub.docker.com/v2/repositories/linkedin/datahub-gms/ The most-pulled artefact in this corpus stopped being updated when the project left the LinkedIn namespace pull_count: 123682069, last_updated: 2024-04-02, date_registered: 2020-05-29
29 Docker Inc acryldata/datahub-gms image metadata source registered 2021-03-09, updated 2026-09-15 2026-09-15 https://hub.docker.com/v2/repositories/acryldata/datahub-gms/ The live image under the new namespace has a twenty-third of the pulls of the frozen one pull_count: 5308970, last_updated: 2026-09-15, date_registered: 2021-03-09
30 PyPI acryl-datahub release history source 2021-04-02 to 2026-09-15 2026-09-15 https://pypi.org/pypi/acryl-datahub/json The spun-out catalogue ships constantly: 1,271 releases in five years first 0.0.2 2021-04-02, latest 1.7.0.11rc2 2026-09-15, 1,271 releases; summary "DataHub ingestion framework and CLI"
31 LinkedIn linkedin/kafka README source branch 3.0-li 2026-09-15 https://raw.githubusercontent.com/linkedin/kafka/3.0-li/README.md LinkedIn runs a patched fork of the engine it invented, and says what the patch set is made of "We run thousands of brokers to deliver trillions of messages per day. We run a slightly modified version of Apache Kafka trunk."; the branch is "Cherry-picked commits from upstream after branch point", "Patches that are on their way upstream but we have deployed internally in the meantime", "Patches that are of no interest to upstream"
32 Maven Central com.linkedin.iceberg:iceberg-core version listing source 2022-06-02 to 2026-08-28 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/iceberg/iceberg-core/ The table-format fork is maintained on two parallel lines at once, four years in 85 versions, oldest 0.11.1.1 2022-06-02, newest 1.5.2.21 2026-08-28; 1.2.0.24 published 2026-08-28 and 1.5.2.21 the same day
33 LinkedIn linkedin/iceberg repository description source listing as of 2026-09-15 2026-09-15 https://github.com/linkedin/iceberg The fork is declared as such rather than presented as a product Repository description: "A home for LinkedIn's changes to Apache Iceberg"
34 LinkedIn OpenHouse README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/openhouse/main/README.md What LinkedIn now builds on top of an open table format is a declarative control plane, not a store "OpenHouse is an open source control plane designed for efficient management of tables within open data lakehouse deployments."; "OpenHouse reconciles the observed state of Tables with the desired state by orchestrating various data services."
35 LinkedIn OpenHouse Tables API specification adr spec v0.1, fetched 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/openhouse/main/docs/specs/catalog.md The control plane's contract is an HTTP API over Iceberg snapshots, which is the seam it owns "OpenHouse Tables APIs v0.1"; PUT /v0/databases/{databaseId}/tables/{tableId}/iceberg/v2/snapshots; body parameter "baseTableVersion": "Base table version to apply the change to"
36 LinkedIn Coral README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/coral/master/README.md The translation layer is the asset: one intermediate representation, many engines "Coral is a SQL translation, analysis, and rewrite engine. It establishes a standard intermediate representation, Coral IR, which captures the semantics of relational algebraic expressions independently of any SQL dialect."; "Coral supports converting HiveQL and Spark SQL to Coral IR, and converting Coral IR to HiveQL, Spark SQL, and Trino SQL."
37 Maven Central com.linkedin.coral:coral-trino version listing source 2021-04-09 to 2026-09-04 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/coral/coral-trino/ The translation layer ships more often than anything else LinkedIn publishes: 343 versions in five years 343 versions, oldest 1.0.41 2021-04-09, newest 2.5.9 2026-09-04
38 LinkedIn Hoptimator README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/Hoptimator/main/README.md The newest in-house system is a planner that turns a materialized view into pipelines across other people's engines "Hoptimator turns SQL into running, multi-hop data pipelines that span Kafka, Flink, Venice, and anything else you plug in."; "You don't write Flink jobs and you don't request topics. The planner figures out the topology from a query."
39 LinkedIn Brooklin README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/brooklin/master/README.asciidoc Change capture became a tenant of a general streaming system rather than its own product "Brooklin is a distributed system intended for streaming data between various heterogeneous source and destination systems"; "Designed for multitenancy, Brooklin can simultaneously power hundreds of data pipelines across different systems"; "Brooklin supports propagating Change Data Capture events from data stores"
40 LinkedIn databus repository source listing as of 2026-09-15 2026-09-15 https://github.com/linkedin/databus The predecessor carries no deprecation notice at all, which is what an unmarked death looks like Repository description: "Source-agnostic distributed change data capture system"; 3.7k stars, 332 commits on master, no archive or deprecation banner
41 LinkedIn Ambry README source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/ambry/master/README.md The second retained store is the blob store, and the design rationale lives in a 2016 paper this corpus could not fetch "Ambry is a distributed object store that supports storage of trillions of small immutable objects (50K -100K) as well as billions of large objects."; "Paper introducing Ambry at SIGMOD 2016"
42 LinkedIn linkedin organisation, public repositories source listing as of 2026-09-15 2026-09-15 https://github.com/orgs/linkedin/repositories?q=sort%3Aupdated&type=public The live estate is 135 public repositories, and the most recently touched are serving, catalogue and AI infrastructure Total 135; Liger-Kernel (updated 2026-09-14), venice (2026-09-14), datahub-gma (2026-09-11), openhouse (2026-09-11), ambry (2026-09-10), Hoptimator (2026-09-04), coral (2026-09-04)
43 LinkedIn linkedin organisation, archived repositories source listing as of 2026-09-15 2026-09-15 https://github.com/orgs/linkedin/repositories?q=archived%3Atrue&sort=updated Only four repositories are archived, so archiving is not how this organisation records a retirement Total 4 archived: rest.li (Jul 22, 2026), prebid.js (Jan 23, 2026), forthic (Jan 1, 2026), test (Aug 29, 2025)
44 Maven Central com.linkedin.feathr:feathr_2.12 version listing source 2022-04-26 to 2023-06-30 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/feathr/feathr_2.12/ The feature store donated to a foundation stopped shipping fourteen months after its first release 78 versions, oldest 0.1.0 2022-04-26, newest 1.0.5-rc5 2023-06-30
45 Feathr / LF AI & Data feathr-ai/feathr repository source listing as of 2026-09-15 2026-09-15 https://github.com/feathr-ai/feathr The README still claims production use at LinkedIn three years after the artifacts stopped "Feathr is a data and AI engineering platform that is widely used in production at LinkedIn for many years and was open sourced in 2022."; "It is currently a project under LF AI & Data Foundation."; 1.9k stars, 140 open issues
46 Maven Central org.apache.samza:samza-core_2.12 version listing source 2017-06-09 to 2023-01-13 2026-09-15 https://repo1.maven.org/maven2/org/apache/samza/samza-core_2.12/ The donated stream processor has not shipped since January 2023 14 versions, oldest 0.13.0 2017-06-09, newest 1.8.0 2023-01-13
47 Maven Central org.apache.gobblin:gobblin-distribution version listing source 2018-06-20 to 2023-06-14 2026-09-15 https://repo1.maven.org/maven2/org/apache/gobblin/gobblin-distribution/ The donated ingestion framework's last release is mid-2023 6 versions, oldest 0.12.0 2018-06-20, newest 0.17.0 2023-06-14
48 Maven Central com.linkedin.gobblin:gobblin-core version listing source 2015-12-02 to 2017-07-21 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/gobblin/gobblin-core/ The handover date is legible: LinkedIn's coordinates stop eleven months before Apache's begin 12 versions, oldest 0.0.0.mavenTest1 2015-12-02, newest 0.11.0 2017-07-21
49 Maven Central org.apache.helix:helix-core version listing source 2013-01-09 to 2026-06-30 2026-09-15 https://repo1.maven.org/maven2/org/apache/helix/helix-core/ The oldest donated component is the one still under the newest system 41 versions, oldest 0.6.0-incubating 2013-01-09, newest 2.0.1 2026-06-30
50 Maven Central com.linkedin.transport:transportable-udfs-api version listing source 2021-05-05 to 2026-07-30 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/transport/transportable-udfs-api/ The portable-function layer is still shipping, on the same cadence as Coral 37 versions, oldest 0.0.65 2021-05-05, newest 0.2.3 2026-07-30
51 Maven Central com.linkedin.dagli:core version listing source 2020-09-24 to 2021-10-05 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/dagli/core/ The in-house machine-learning framework never left beta and stopped in 2021 9 versions, oldest 14.0.0-beta1 2020-09-24, newest 15.0.0-beta9 2021-10-05
52 Maven Central com.linkedin.tony:tony-core version listing source 2018-10-03 to 2022-05-26 2026-09-15 https://repo1.maven.org/maven2/com/linkedin/tony/tony-core/ In-house distributed training on YARN ran four years and stopped 53 versions, oldest 0.1.2 2018-10-03, newest 0.5.3 2022-05-26
53 npm dustjs-linkedin package metadata source 2012-05-18 to 2021-12-29 2026-09-15 https://registry.npmjs.org/dustjs-linkedin A decade-long fork of somebody else's template engine, still installable, last touched 2021 description "Asynchronous templates for the browser and server (LinkedIn fork)"; 55 versions, first 0.4.0 2012-05-18, last 3.0.1 2021-12-29
54 Go module proxy github.com/linkedin/burrow version list source 28 tagged versions, latest v1.9.6 2026-09-15 https://proxy.golang.org/github.com/linkedin/burrow/@v/list The Kafka lag monitor is still a published Go module, and consumer lag is still treated as a first-class signal 28 versions listed, highest v1.9.6
55 LinkedIn Venice contributing index source current 2026-09-15 2026-09-15 https://raw.githubusercontent.com/linkedin/venice/main/docs/contributing/index.md The project publishes its architecture notes and proposal process in the repository rather than only on its website Sections "Architecture" (Write Path, Java Internals, Router API) and "Proposals" ("Venice Improvement Proposals (VIPs)")

What is missing, and what that means

  • No postmortems written by LinkedIn. The incident evidence in this guide is the GitHub Advisory Database, which records defects and their fixes but not blast radius, detection time or customer impact. Every "blast radius" line in the failure section is therefore stated as what the defect made possible, not as what happened to a named deployment.
  • No papers and no talks. Ambry's SIGMOD 2016 paper and the Pinot and Venice conference material exist; this session could not reach them. A reader with unrestricted access should start there.
  • No engineering-blog posts. The motivations quoted here are the ones LinkedIn wrote into repository files. Where the repository points at a blog post for the reasoning, the guide names the gap instead of filling it.
  • Nothing from inside. Espresso, LinkedIn's online document store, and the feed and search serving systems have no public repository at all, so this guide can describe the derived-data half of the read path and not the primary-data half.