Deciding what to stop building: ten years of Spotify, read from its own archive notices

The territory

A company that wrote its own platform has to decide, component by component, when to stop paying for it. That decision is normally private. Spotify's is partly public, because the company writes the reason into the repository on the way out.

163

of 289 public repositories archived

GitHub org listing, 2026

700k

events per second on the stack they left

Google Cloud blog, 2016

10k+

production workflows running when the repo was discontinued

spotify/styx, archived 2023

security advisories published on one day

Backstage advisories, 2026-08-28

The public record of a platform is not a description of the platform. It is a description of what a company was willing to publish, which is a different thing, and the gap between them is where most of the useful reading sits. Spotify is an unusually good subject for this because it published a great deal between 2013 and 2019, stopped publishing infrastructure almost entirely after 2020, and left behind dated notices explaining several of the retirements. Read in order, those notices are a decision log that no one intended to write.

The shape of the decade is blunt. In 2014 Spotify was running a container orchestrator it had written, a Java service framework it had written, a time series database it had written, a service discovery library, a secret distribution service, a batch scheduler and a vector search library, and it published all of them. In 2026 the orchestrator, the time series database, the discovery library, the secret service, the model server and the batch scheduler are archived, the service framework has been withdrawn from public view while remaining in production, and the newest infrastructure Spotify publishes is a feature flag resolver in Rust, first released to crates.io in January 2026. The one platform component that grew rather than shrank is the software catalogue, which Spotify open sourced in March 2020, gave to a foundation in September 2020, and is still funding in 2026.

The finding that changes how you read any of this: archiving a repository at Spotify is a publishing decision, not a retirement decision, and the two are routinely in different years. The Styx scheduler's repository was discontinued on 12 July 2023 with the README still stating that it ran more than 10,000 production workflows. Apollo, the service framework, was archived in September 2021 with a notice saying in plain words that development would continue internally at Spotify. An architect who reads a company's open source estate as a proxy for what that company runs will systematically undercount, and the direction of the error is always the same.

Scope

This guide covers the platform layer that leaves a public trace: orchestration, service frameworks, observability, batch and streaming data, model serving, vector search, and the developer portal. It does not cover Spotify's audio serving path, its recommendation systems, its storage internals, or any part of the architecture the company never published. It contains no outage postmortem, because none is reachable, and no conference talk or paper, for the same reason. The evidence limits are set out in full in section 06.

Figure 1 · Where each capability went

Every in-house infrastructure component in the public record was replaced by a managed service from a single vendor, and the one component that survived the decade is the layer that describes the others. Reconstructed from the archived repository listing, the Styx README and the 2016 event delivery account.

Diagram source

Figure 2 · The exits, in the order they were recorded

Retirement is a batch activity and it lags the technical decision by years: Helios stopped being used when Spotify moved to Kubernetes, and was archived in September 2024. Dates from the repository notices listed in section 06.

Diagram source

How it is actually built

Three layers, and only one of them is still written in house. The interesting part is which one.

Reconstructing the shape from the repositories rather than from a diagram someone drew gives a cleaner answer than the usual platform picture. There is a rented runtime, where the artefact is configuration rather than code and nothing about it appears in a public repository. There is a thin layer of glue that is still code, because no vendor ships it: operators, job submitters, pipeline libraries and export tools. And there is a description layer, the catalogue of what exists and who owns it, which is where Spotify has put its platform engineering money since 2020.

The rented layer is legible only by its absence. Spotify's public repositories once held a DNS library for service discovery, dns-java, and a secret distribution service, talos; both are archived with 2024 dates. They were followed by gordon, an event-driven Cloud DNS registration service, which is archived too. Nothing public replaced either one, because the replacement is a managed service. This is the single most important thing to understand about reading an estate like this: the more successfully a company rents its infrastructure, the less of its architecture is visible, and the emptiness is the evidence.

Figure 3 · Reference architecture of a rented platform

The glue layer is thin and the runtime layer is bought, so the catalogue becomes the only place where the system as a whole is described. Components attributed to flink-on-k8s-operator, dbeam, scio and the catalogue format ADR.

Diagram source

The catalogue is a port, not an invention

The second architecture decision record in the Backstage repository states that the format was carried over from an internal system: "Internally at Spotify, a homegrown software catalog system is used heavily and forms a core part of Backstage and other important pieces of the infrastructure." The entity format borrows its shape from Kubernetes objects, which is why anyone who has written a Kubernetes manifest can read a catalogue file without being taught.

Source: ADR002

The glue is where the reversals happen

Having moved batch and streaming onto a managed runner in 2016, Spotify now maintains a fork of a Google operator for running Flink and Beam on its own Kubernetes, described as "an open source fork of GoogleCloudPlatform/flink-on-k8s-operator with several new features and bug fixes". Renting the runtime is not a one-way door, and the return trip is usually driven by a workload the managed product prices or schedules badly.

Corroborated by: scio, still active on Beam

Three states, not two

The dbeam README declares a state most estates lack a word for: "Mature, maintained and used in production since August 2017" with "No major features or development planned". Finished is not abandoned, and labelling it removes the most common reason a dependency gets rewritten, which is that nobody can tell the difference between quiet and dead.

Contrast: heroic, frozen with no successor named

Inside the surviving component, the architecture has been rebuilt twice while live, and both rebuilds are readable in the registry rather than in an announcement. The original backend core package, @backstage/backend-common, published its first version on 30 April 2020 and its last on 17 September 2024, after 1,347 releases, and now carries a registry deprecation notice telling consumers to follow per-export migration instructions. The frontend has gone the same way: the first public package, @backstage/core, was published on 26 March 2020 and superseded within fifteen months. A package registry is an underrated source for this kind of archaeology, because it records what shipped and when, which is frequently not what the roadmap says.

The plugin boundary is the other structural change. Around sixty community plugins were moved out of the core monorepo into a separate repository where "each plugin workspace has its own changesets and isolated releases". That is a scaling decision about maintainer attention rather than about software, and it is the same decision any platform team makes when the number of optional components passes the number of people who can review them.

The decisions that matter

Each fork below is recorded somewhere in the public estate, usually in a sentence someone wrote while closing something down.

Decision: keep the orchestrator you wrote, or move to the one everyone else uses?

Chosen

Move to Kubernetes and stop developing Helios
The notice states the original condition and its expiry in one sentence: Helios "was created when there were no open source container orchestration frameworks"

Rejected

Continuing an orchestrator with 2,458 commits and published Java artifacts up to version 0.9.283
Rejected on ecosystem, not on capability: nothing in the notice says Helios stopped working

Flips when

The upstream does not yet cover your scheduling constraint, which is why the same company later maintains its own Flink operator on top of the upstream it adopted

Decision: run the event bus, or rent it?

Chosen

Replace a self-run Kafka 0.8 and HDFS pipeline with managed messaging and a managed batch and streaming runner, from 2016
The system being replaced was carrying 700,000 events per second, and the replacement was load tested at 2,000,000 messages per second

Rejected

Scaling the existing Kafka, HDFS and Crunch-MapReduce stack in place
The stated design goals for the successor were automation, reliability, persistence and logical event type separation, which is a list of operational complaints

Flips when

Egress or per-message pricing dominates your bill, or when the managed product's delivery semantics do not match your consumer model; note that the same team kept writing its own export and deduplication code afterwards

Decision: keep publishing the thing you still run?

Chosen

Withdraw the repository while keeping the system: Apollo's notice says development "will continue internally in Spotify"; Styx's says "We decided to discontinue the Styx oss repo" while the README still described more than 10,000 production workflows

Rejected

Continuing to carry the cost of a public interface for software with, in Apollo's words, "very little (if any) use outside Spotify"

Flips when

External contribution exceeds the review cost, which is the test the same company applied in the other direction when it donated its catalogue to a foundation in 2020

Figure 4 · The question behind every archive notice

Each terminal state in this tree appears verbatim in Spotify's estate, and the two on the right are the ones most decision frameworks omit. Derived from the notices on helios, apollo, styx and dbeam.

Diagram source

The decisions inside the surviving component are documented far better than the decisions that killed the others, and the contrast is instructive. Backstage keeps an architecture decision record log whose first entry is the decision to keep one, complete with the risk that kills most such logs: "People stop adding ADRs to the log and context gets lost." On top of that sits a proposal process modelled on Kubernetes enhancement proposals, in which merging a proposal means it "is approved for implementation and has an owner that is responsible for said implementation". Tying approval to a named owner is the part worth copying. It is also the part that explains why one of the two largest proposals has not shipped.

Decision	Chosen	Rejected	Because	Flips when	Evidence
Second database engine for the portal	PostgreSQL and SQLite only	Production MySQL support in clustered deployments	Maintainer judged the parallel migration logic "not maintainable long-term" and questioned the premise: "why support clustering of this data at all?"	An adopter funds a standing maintainer for the second engine, not a single pull request	PR 20409, closed unmerged 2024-01-31
Where optional plugins live	A separate repository with per-workspace releases	Keeping roughly sixty community plugins in the core monorepo	Release coupling and review load scale with plugin count, not with core changes	Plugin count falls below what your reviewers can hold, or you adopt dynamic loading and stop shipping plugins in the build at all	community-plugins
How plugins reach a running portal	Build-time composition, still the shipped model	Runtime-loaded dynamic plugins	Proposed in January 2024 on adoption grounds: an installation "requires quite a lot of care to maintain"; still marked provisional	Your users cannot run a build pipeline, which is exactly why a downstream distribution shipped dynamic plugins first	BEP 0002, RHDH
Default trust posture	Deny by default, with plugins opting out per path	The previous model of an unauthenticated instance protected by something outside it	Adopters were deploying an internal tool on the internet: the goal was to make it "safe to deploy Backstage directly towards the internet"	Never, once the software runs somewhere you do not control the network	BEP 0003, 2024
Where ownership data lives	Behind a service call	Claims embedded in the user token	Tokens had grown "large enough to have an impact on performance and reliability", against a 4kb cookie limit	Your identity payload is small and stable, which is a property that decays silently as the catalogue grows	BEP 0003
Core asynchronous job service	Not yet: closed unmerged after two months of review	A queue service with memory, Redis, PostgreSQL, Kafka and SQS adapters	Maintainer wanted a first internal consumer: "something that could conceivably replace some of the catalog processing queues"	An existing subsystem can be migrated onto the new abstraction in the same change	PR 32623, 2026
Vector search implementation	A new library on hierarchical navigable small world graphs	Continuing the memory-mapped tree index shipped since 2014	Stated as recall, not memory: the successor is "like Annoy, but with much higher recall"	Your index must be shared read-only across processes on one machine, which is what the older design optimised for	voyager, annoy releases
Who sets the roadmap of the donated component	Community input, collected in a public issue	The originating company setting direction alone	"an open invitation to help shape where we focus our collective efforts in 2026"	You need a decision the community will not fund, at which point a vendor distribution makes it instead	Roadmap issue, opened 2025-12-09

What broke in production

There is no outage postmortem in this corpus. What exists instead is a security advisory record, two bug reports from adopters, and a build file. Read carefully, those four sources describe four distinct failure classes.

The honest framing matters here. Spotify publishes no incident reviews that this session could reach, and the Backstage project publishes advisories rather than postmortems, so blast radius is almost always unknown: an advisory says what could happen, not what did. Every card below states which it is. The value is not in the individual defects, it is in the shape of the record, and the shape is unambiguous. Of the 69 published advisories, the large majority name two features, and they are the two features where the portal executes text that a user committed to a repository.

Advisory pattern

The templating engine keeps being an execution boundary

AssumptionA template or a documentation config committed by a colleague is data, because everyone who can commit is trusted.

What happenedThe same class recurs for five years: remote code execution through v1beta3 templates in November 2021, an insecure sandbox in the scaffolder in June 2023, arbitrary code execution through MkDocs hooks in January 2026, and a further batch of scaffolder and TechDocs findings in August 2026.

Blast radiusUnknown in every case; advisories describe reachability, not incidents. Severity ranges from moderate to high across the four named findings.

FixStructural rather than incremental: configuration sanitisers, allowlists on config keys, and a default-deny authentication policy for every plugin route.

Design ruleAny feature that evaluates a file from a repository is an execution boundary with the portal's own credentials behind it. Treat the golden-path templating engine as a code execution service and give it its own identity, not the portal's.

SourceAdvisories 2021 to 2023, 2024 to 2026

Advisory

Execution data leaked across users through task logs

AssumptionTask logs are diagnostics, so any authenticated user may read another user's task.

What happened"An authenticated Backstage user who can read another user's Scaffolder task may receive internal execution data. In deployments where that data contains credentials for an external service, this may permit disclosure and unauthorized changes in that external service."

Blast radiusRated critical, CVSS 9.6, published 28 August 2026. Actual exploitation is not stated.

FixPatched in four maintained release lines at once, and the interim guidance is to scope task read permissions by ownership.

Design ruleA log written by a process holding credentials inherits that process's sensitivity. Decide the read permission on a task log before you decide its retention.

SourceGHSA-g2v8-7jhw-pp8p, 2026-08-28

Adopter report

The catalogue's scale ceiling was found by someone else

AssumptionAn entity count that works for the maintainers works for adopters, and catalogue reads are cheap because entities are small.

What happenedOne adopter reported that at roughly 9,000 entities the portal behaved normally and at 21,000 entities, of which 14,000 were users, "every fetch request will timeout". A second reported facet queries taking one to three seconds and occasionally six, with a 6.5 second catalogue page load, at just over 2,000 components.

Blast radiusPortal unusable for that installation. Both reports were closed as not planned, in April 2025.

FixNone recorded in either thread. The second report traced the cost to the facet query itself, on both PostgreSQL and SQLite.

Design ruleLoad test the catalogue at the entity count your identity provider will produce, not the count your services will. User and group ingestion is what pushes an internal catalogue past its tested range, and it arrives on the day you connect the directory.

SourceIssue 26665, Issue 27907

Repository record

Archiving ends maintenance, not deployment

AssumptionA deprecation notice retires the software, so nobody has to think about it again.

What happenedHeroic was frozen on 27 March 2021 with the notice that there would be no new versions. Its final build file pins log4j-core 2.13.3. On 10 December 2021, 258 days later, CVE-2021-44228 was published against exactly that range, at CVSS 10.0. The container image remains on a public registry and has been pulled 6,462 times.

Blast radiusUnknown and unknowable from outside: no public account says who was still running Heroic in December 2021. That is itself the point.

FixNone upstream is possible; the repository is read-only. Any remaining operator had to fork or replace.

Design ruleAn archive notice transfers the patching obligation to every operator silently, at an unknown future date. Keep a register of archived upstreams you still run and treat the archive date as the start of a countdown, not the end of a story.

Sourceheroic build.gradle, GHSA-jfh8-c2jp-5v3q, image pull count

Repository record

A live dependency pinned by a dying project

AssumptionInternal consumers track the latest version of an internally maintained library.

What happenedThe docker-client deprecation notice says it plainly: "Spotify no longer uses recent versions of this project internally. The version of docker-client we're using is whatever helios has in its pom.xml." The effective version of a shared library was set by the build file of a project the company had already stopped developing.

Blast radiusNot quantified. The library has 1.4k stars and 534 forks, so the pinning behaviour propagated outward as well.

FixArchived on 21 March 2022, in the same sweep as dockerfile-maven and spydra.

Design ruleVersion pins are a dependency graph of their own. When you retire a system, publish the versions it pinned, because somewhere a live service is inheriting them.

Sourcespotify/docker-client

Design record

An internal tool that was deployed on the internet

AssumptionThe portal sits inside a corporate network, so authentication is the integrator's problem and the shipped default can be open.

What happenedFour years after open sourcing, the maintainers wrote a proposal to invert that default, stating the goal as making it "safe to deploy Backstage directly towards the internet" and noting that until then adopters had to either set up protection themselves "or not do so and risk exposing your instance to malicious actors".

Blast radiusUnknown across the roughly 290 organisations that list themselves as adopters.

FixA default-deny policy on every plugin route with explicit per-path opt-out, cookie authentication with a limited-scope token, and an on-behalf-of flow replacing forwarded user tokens.

Design ruleSoftware written inside a trust boundary carries that boundary as an unstated default. When you open source an internal tool, the first architectural task is to find every place the design assumed the network was friendly.

SourceBEP 0003, created 2024-01-28

Figure 5 · The path the advisories keep describing

One repository write reaches every system the portal holds credentials for, which is why the templating and documentation features dominate the advisory record. Reconstructed from the advisory listing and GHSA-g2v8-7jhw-pp8p.

Diagram source

Figure 6 · Five ways software leaves an estate

Only two of these five states are legible from outside without reading the notice text, which is why an estate census that counts archived repositories alone gets the answer wrong. States drawn from the notices cited in section 06.

Diagram source

Numbers you can plan against

Everything quantitative in this corpus, with the date attached. Most of it is counting, because counting is what a repository record supports.

Metric	Value	At	Context	As of	Source
Public repositories, archived	163	Spotify org	Against 126 active, so 56% of the estate is read-only	2026-09	GitHub
Events per second before the 2016 migration	700,000	Spotify event delivery	Kafka 0.8, HDFS, Crunch-MapReduce	2016-03	Google Cloud blog
Messages per second in the replacement load test	2,000,000	Managed messaging	Vendor-hosted account of a Spotify test	2016-03	Google Cloud blog
Production workflows at repository discontinuation	10,000+	Styx	Batch jobs in containers on Kubernetes	2023-07	spotify/styx
Published security advisories	69	Backstage	Seven listing pages, first dated June 2021	2026-09	Advisory listing
Advisories published on a single day	37	Backstage	28 August 2026, concentrated in scaffolder and TechDocs	2026-08	Advisory listing
Highest published severity	9.6 CVSS	Backstage scaffolder	Cross-user read of task execution data	2026-08	GHSA-g2v8-7jhw-pp8p
Entity count where an adopter saw timeouts	21,000	Backstage catalogue	Of which 14,000 users; 9,000 entities behaved normally	2024-09	Issue 26665
Facet query latency at about 2,000 components	1 to 6 s	Backstage catalogue	Catalogue page load measured at 6.5 s	2024-11	Issue 27907
Published versions of the deprecated backend core	1,347	@backstage/backend-common	2020-04-30 to 2024-09-17, now deprecated in the registry	2026-09	npm registry
Published versions of the scaffolder backend	2,144	@backstage/plugin-scaffolder-backend	Latest 4.1.0 published on the day of the advisory batch	2026-08	npm registry
Listed adopter organisations	~290	Backstage	Self-reported table rows, Spotify included	2026-09	ADOPTERS.md
Luigi releases since first publication	87	PyPI	2012-10-21 to 2026-05-07, still shipping	2026-05	PyPI
Gap between Annoy's last release and Voyager's first	45 days	PyPI	Derived: 2023-06-14 to 2023-07-28	2023-07	PyPI
Queries per day served by the vector index	hundreds of millions	Voyager at Spotify	Stated in the README, not measured externally	2024-12	spotify/voyager
Pulls of the archived time series database image	6,462	Docker Hub	Image last updated 2021-03-26	2026-09	Docker Hub
Pulls of a container image last built in 2018	16,791	Docker Hub	The Luigi scheduler image	2026-09	Docker Hub
iOS builds collected by the build metrics platform	~1,000,000	XCMetrics	With over 10 billion build steps, across all Spotify iOS apps	2026-09	spotify/XCMetrics
Downloads of the 2026 Rust feature flag resolver	72,115	crates.io	First published 2026-01-19, 25 versions	2026-09	crates.io

Read these carefully

Three of these are measured by someone with no incentive to flatter: the adopter latency numbers and the entity counts come from bug reports. The 2016 throughput figures are a vendor's account of a customer's system and should be treated as claims. The query volume for the vector index is a README statement with no measurement behind it. The counts of repositories, advisories, versions and pulls are reproducible by anyone who repeats the queries in section 08, and they will drift, so re-run rather than cite. The most important number in this section is the one that does not exist: nobody has published how many entities Spotify's own catalogue holds, which means the only public scale evidence for the catalogue comes from installations far smaller than the one it was built for.

The evidence wall

Every source behind this page, graded. Filter by kind. Two tiers are missing entirely and that is reported rather than hidden: there are no conference talks and no peer-reviewed papers here, because this session's network policy reached code hosts, package registries and one cloud vendor's blog, and nothing else.

Source Spotifyarchived 2024-09

spotify/helios

The in-house container orchestrator, public from May 2014 and archived in September 2024. The sunset notice is the clearest build-versus-adopt statement in the estate: it names the condition that justified building, and the event that ended it.

Carry forwardWrite the expiry condition into the README when you build something because nothing else exists. Somebody will need it years later.

github.com/spotify/helios

Source Spotifyarchived 2021-09

spotify/apollo

The Java microservice framework, archived with a notice saying development continues inside the company and that the project has "very little (if any) use outside Spotify". The archive is about the audience, not about the software.

Carry forwardWithdrawing a repository is a supportable decision, but say so explicitly or readers will record your system as dead.

github.com/spotify/apollo

Source Spotifyarchived 2023-07

spotify/styx

A batch scheduler running jobs in containers on Kubernetes against managed datastores. Discontinued as a public repository while the README described more than 10,000 production workflows, and while the older library it was built to orchestrate carried on shipping.

Carry forwardThe replacement can die before the thing it replaced. Check which of your internal tools is actually load bearing before you retire the other one.

github.com/spotify/styx

Source Spotifyarchived 2021-03

spotify/heroic

The in-house time series database built on Bigtable, Cassandra and Elasticsearch, frozen in March 2021 with no successor named in public. Its final build file is the evidence for the abandonment-tail failure in section 04.

Carry forwardA freeze notice is a statement about the maintainers, not about the deployments. Inventory the deployments.

github.com/spotify/heroic

Source Spotify2021 build file

heroic build.gradle on master

Pins log4j-core 2.13.3 and a 2019-era dependency set including Elasticsearch 7.5.0, Jackson 2.9.9 and slf4j 1.7.21. Read next to the Log4Shell advisory it dates the moment an archived project became an unpatchable one.

Carry forwardDiff the dependency pins of every archived upstream you still run against the advisory database once a quarter. It is a ten-line script.

raw.githubusercontent.com/spotify/heroic

Postmortem GitHub Advisory DB2021-12-10

GHSA-jfh8-c2jp-5v3q, CVE-2021-44228

The Log4Shell advisory, CVSS 10.0, affecting log4j-core versions at or above 2.13.0 and below 2.15.0. Heroic's pinned 2.13.3 falls inside that range, 258 days after the repository went read-only.

Carry forwardArchive dates and disclosure dates are independent random variables. Plan for the case where the second follows the first.

github.com/advisories/GHSA-jfh8-c2jp-5v3q

Source Spotifyarchived 2022-03

spotify/docker-client

Deprecated with the admission that the internally used version was whatever the already dying orchestrator pinned in its build file. Archived in a sweep with dockerfile-maven and spydra in March 2022.

Carry forwardWhen one project pins another, retiring the first freezes the second for everyone downstream of it.

github.com/spotify/docker-client

Source Spotify2026-09 listing

Archived repository listing, 163 entries

The estate census: 163 archived against 126 active. The dates cluster, with eight archives carrying a late March 2022 timestamp, which shows retirement being run as a campaign rather than as a continuous practice.

Carry forwardRun the same census on your own organisation once a year; the clusters tell you when someone last had time to clean up.

github.com/orgs/spotify/repositories

Source Spotify2026-09 listing

Active repository listing, 126 entries

What Spotify publishes now: feature flag SDKs, a Rust flag resolver, audio libraries, a Flink operator and an agent tooling plugin. No orchestrator, no service framework, no metrics system.

Carry forwardThe genre of a company's open source output tells you which layer it still considers differentiating.

github.com/orgs/spotify/repositories

Source Spotifyactive

spotify/flink-on-k8s-operator

A maintained fork of a Google-originated Kubernetes operator for Flink and Beam. The clearest evidence that renting a runtime is not a terminal state: some workloads come back to code you maintain.

Carry forwardBudget for the fork. Adopting upstream converts build cost into maintenance cost rather than removing it.

github.com/spotify/flink-on-k8s-operator

Source Spotifyproduction since 2017-08

spotify/dbeam

Declares itself finished rather than abandoned: mature, in production since August 2017, with no further features planned. The only repository in this estate that names that state explicitly.

Carry forwardAdd a "finished" label to your own repository taxonomy. It is the honest answer for most internal libraries.

github.com/spotify/dbeam

Source PyPI2012 to 2026

Luigi release history

Eighty-seven releases from October 2012 to May 2026, still maintained by Spotify's data team. The oldest tool in the corpus outlived the Kubernetes-native scheduler built to run its jobs.

Carry forwardLongevity in a registry beats architecture fashion as a predictor of what will still be running.

pypi.org/pypi/luigi

Source PyPI2014 to 2023

Annoy release history

First release March 2014, last release 14 June 2023. The handover to its successor is visible as a 45 day gap between the two projects' final and first releases.

Carry forwardRegistry dates give you the real end of life; the archive flag gives you the paperwork date, often years later.

pypi.org/pypi/annoy

Source Spotify2023 to 2024

spotify/voyager

The replacement vector index, justified on recall rather than on memory layout, and described as serving hundreds of millions of queries a day. A rare in-house component that was replaced by another in-house component rather than by a product.

Carry forwardKeep building where the state of the art moves faster than the vendors, which for vector search it did.

github.com/spotify/voyager

Eng blog Spotify via Google Cloud2016-03-03

Why Spotify migrated its event delivery system from Kafka to Pub/Sub

The only pre-2020 narrative source reachable here. Gives the scale of the system being replaced, 700,000 events per second on Kafka 0.8 with HDFS and Crunch-MapReduce, and the design goals of the successor.

Carry forwardVendor-hosted accounts of a customer migration carry real numbers about the old system and marketing about the new one. Use the first half.

cloud.google.com/blog

Eng blog Spotify via Google Cloud2016-03-11

Spotify's experiments with stream processing on Cloud Dataflow

Names what was being retired, Kafka with HDFS and Crunch-MapReduce, and what the team had to learn instead: windows, watermarks and late-arriving data. The operational surface moved rather than disappeared.

Carry forwardManaged runners replace operations with semantics. Budget the learning, not just the migration.

cloud.google.com/blog

Source Maven Central2026-09

com.spotify artifact directory

The Java-era estate in one listing: apollo modules, crtauth, dns, docker-client, crunch, dataproc-java-submitter, bigtable-autoscaler, heroic, scio, dbeam. A published artifact survives the repository that built it.

Carry forwardPackage registries are the most durable record of an architecture. They do not get archived.

repo1.maven.org/maven2/com/spotify

Decision record Backstage2020

ADR001: Architecture Decision Record log

The first decision is to keep a decision log, and it records its own risk: "People stop adding ADRs to the log and context gets lost." Fifteen ADRs exist six years later, and the heavier decisions moved to a separate proposal process.

Carry forwardAn ADR log decays into style guidance unless a second, heavier process exists for architecture-sized changes.

raw.githubusercontent.com ADR001

Decision record Backstage2020

ADR002: Default software catalog file format

States that the format is carried over from an internal Spotify catalogue that "forms a core part of Backstage and other important pieces of the infrastructure", and that it borrows the Kubernetes object shape.

Carry forwardBorrow the entity shape your engineers already read. Familiarity is a migration cost you can avoid for free.

raw.githubusercontent.com ADR002

Decision record Backstageprocess

Backstage Enhancement Proposals

A Kubernetes-style proposal process where merging means approved and owned. Fourteen proposals exist, with statuses including provisional, implementable, implemented, deferred and rejected.

Carry forwardMake the owner a merge condition. A design with no owner is a backlog item wearing a document.

raw.githubusercontent.com BEP README

Decision record Backstage2024-01-28

BEP 0003: Auth architecture evolution

Inverts the default trust model so that an instance is safe to expose directly, adds an on-behalf-of flow instead of forwarding user tokens, and moves ownership claims out of the token because tokens had grown large enough to affect reliability.

Carry forwardToken payloads grow with your catalogue. Put identity attributes behind a service before the cookie limit puts them there for you.

raw.githubusercontent.com BEP 0003

Decision record Backstage2024-01-17, provisional

BEP 0002: Dynamic frontend plugins

Proposes loading plugins at runtime so an installation can be maintained without a codebase. Two and a half years later it is still marked provisional, while a downstream vendor distribution ships the capability.

Carry forwardWhen a proposal stalls, look downstream. Someone with a commercial deadline has usually shipped a version of it.

raw.githubusercontent.com BEP 0002

Postmortem Backstage2021 to 2026

Security advisory listing, 69 entries

Seven pages of advisories from June 2021 to September 2026, of which 37 carry a single date, 28 August 2026. The distribution across components is the finding: templating and documentation rendering dominate.

Carry forwardA one-day batch of dozens of advisories is the signature of an audit, not of an attack. Read the component distribution, not the count.

github.com/backstage/backstage/security/advisories

Postmortem Backstage2026-08-28

GHSA-g2v8-7jhw-pp8p, sensitive information exposure in Scaffolder

Critical, CVSS 9.6: an authenticated user reading another user's task may receive internal execution data, including credentials for external systems in some deployments. Patched across four release lines at once.

Carry forwardScope task log reads by ownership on day one. The convenience of shared logs is worth less than the credential blast radius.

GHSA-g2v8-7jhw-pp8p

Postmortem Backstage2021-06 to 2023-02

Advisory listing, the first two years

The earliest page of the record already shows the pattern: TechDocs configuration path traversal and content sanitisation bypasses in June 2021, template remote code execution in November 2021, catalogue cross-site scripting in February 2023.

Carry forwardFailure classes in a portal are stable across years. If you adopt one, put your review effort where its advisories already cluster.

Advisory listing, page 7

Postmortem Backstage2023-06 to 2026-03

Advisory listing, the middle years

Includes an insecure sandbox in the scaffolder in June 2023, storage bucket directory traversal in TechDocs in September 2024, and arbitrary code execution through MkDocs hooks in January 2026.

Carry forwardSandboxing user-supplied configuration inside your own process keeps failing. Run it somewhere you can afford to lose.

Advisory listing, page 6

Source Backstage2024 to 2025

Issues 26665 and 27907, catalogue scale

Two adopter bug reports with numbers: timeouts at 21,000 entities of which 14,000 are users, and facet queries costing one to six seconds at just over 2,000 components. Both closed as not planned.

Carry forwardSearch the issue tracker for the scale you plan to run at before you adopt. The answer is usually there, filed by someone who got there first.

Issue 26665

Source Backstage2024-01-31

PR 20409, continued MySQL support, closed unmerged

A substantial contribution adding MySQL support for clustered deployments, rejected on maintenance cost with the premise itself questioned. The most useful rejected pull request in the repository for anyone planning a portal deployment.

Carry forwardCheck which database the project actually tests before you promise your platform team the one you already run.

PR 20409

Source Backstage2026

PR 32623, core queue service, closed unmerged

A queue service with five backend adapters, reviewed for two months and closed without merging, in part because no existing subsystem was migrated onto it in the same change.

Carry forwardShip a new core abstraction with its first consumer attached, or expect to be asked for one.

PR 32623

Source npm2020 to 2026

Registry metadata for the Backstage packages

Dates the open sourcing at 26 March 2020, the backend core's deprecation at September 2024 after 1,347 versions, and the remediation release of the scaffolder backend at 28 August 2026, the same day as the advisory batch.

Carry forwardRegistry timestamps corroborate or contradict a project's own narrative, and they are machine readable.

registry.npmjs.org

Case study CNCF2020 to 2024

CNCF technical oversight issues for Backstage

Sandbox acceptance merged 8 September 2020, incubation proposal merged 15 March 2022, and a governance review opened on 13 September 2024 and still open. Donation is a process with recurring obligations.

Carry forwardFoundation donation moves governance cost rather than removing it, and the review cadence is public.

github.com/cncf/toc

Case study Red Hatactive

redhat-developer/rhdh

An enterprise distribution of the same portal, formerly janus-idp/backstage-showcase, that ships dynamic plugin loading and a curated plugin set. The divergence point between upstream and downstream is exactly the stalled proposal.

Carry forwardWhere a vendor distribution differs from upstream, it is telling you what enterprise adopters could not live without.

github.com/redhat-developer/rhdh

Case study Backstage2026-09

ADOPTERS.md

Roughly 290 self-reported adopters, from retailers and airlines to banks and scooter companies. Spotify's own entry describes the portal as the "main interface towards all of Spotify's infrastructure and technical documentation".

Carry forwardA self-reported adopter list is weak evidence of success and strong evidence of who to ask for operational detail.

ADOPTERS.md

Case study Spotify2026-09

spotify/XCMetrics

The build metrics platform for the iOS estate, with one of the few published internal scale figures: close to a million builds and over ten billion build steps collected across all Spotify iOS applications.

Carry forwardBuild telemetry is the cheapest platform investment that produces defensible architecture decisions later.

github.com/spotify/XCMetrics

Source crates.io2026-01

confidence_resolver crate

First published 19 January 2026, 25 versions and 72,115 downloads by September. The newest infrastructure component Spotify publishes is a feature flag resolver in Rust, not a platform runtime.

Carry forwardWatch which language a company chooses for new systems code. It dates the shift more precisely than any announcement.

crates.io

Source Backstage2024 onward

backstage/community-plugins

Roughly sixty plugins moved out of the core repository into per-plugin workspaces with isolated releases, which is a decision about reviewer attention rather than about code. The repository carries 9,702 commits and 697 forks.

Carry forwardSplit optional components out when their count passes the number of people who can review them, and give each one its own release cadence.

github.com/backstage/community-plugins

Build a miniature, then productionise it

The skill this guide teaches is reading an estate you do not work in, and then applying it to the one you do. Every rung below is a weekend or less.

Census one organisation's repositories

Pick any company with a public estate and list every repository with its archived flag, description and last push date. The listing pages give you this without an API token.

Done when: you can state the archived-to-active ratio and name the three months with the most archives. Teaches: retirement is a campaign, and the campaign dates tell you when the platform team last had slack.

Extract the archive notices into a decision log

Fetch the README of every archived repository and keep the ones whose first section is a notice. Record the stated reason, the replacement if named, and the date.

Done when: you have a table of reasons, and can sort them into replaced upstream, withdrawn to internal, finished, and no reason given. Teaches: most companies leave a decision record without meaning to, and it is the only one that survives reorganisations.

Date the real ending with a package registry

For each project, pull the release history from the registry it publishes to and compare the last release date with the archive date.

Done when: you can show the median gap between last release and archive flag for that estate. Teaches: the paperwork date is not the engineering date, and citing the wrong one makes your timeline wrong by years.

Census the advisories of a component you are about to adopt

Page through its published security advisories and tag each by affected component and date. Plot the count by component.

Done when: you can name the two components that produce most of the findings and explain what they have in common. Teaches: failure classes are stable; the defect list is a map of where the trust boundary really is.

Read the rejected pull requests before you commit to the tool

Sort closed and unmerged pull requests by comment count, read the top ten, and note every capability the maintainers declined and why.

Done when: you have a list of things the project will not do, in the maintainers' words, that you can take to a design review. Teaches: a project's boundaries are documented in its rejections, never in its README.

Build the deprecation clock for your own estate

For every third-party dependency your services run, resolve the upstream repository and record whether it is archived and when. Alert on newly archived upstreams, and diff their final pinned versions against the advisory database on a schedule.

Done when: a newly archived upstream raises a ticket automatically, with its pinned transitive versions attached. Teaches: the failure in section 04 that has no fix once it happens, and a countermeasure that costs one scheduled job.

Put ownership in a catalogue and test it at directory scale

Stand up a software catalogue, ingest your services from version control, then ingest your whole identity directory and measure list and facet queries at the entity count that produces.

Done when: you have a latency curve against entity count, and know where your installation stops being interactive. Teaches: the scale limit adopters keep reporting arrives from user ingestion, not from service growth.

Keep hunting

These are the queries that produced this page, in the order they were run. They work against any organisation with a public estate, and they do not need an engineering blog to exist.

Census the estate

https://github.com/orgs/<org>/repositories?q=archived%3Atrue
https://github.com/orgs/<org>/repositories?q=archived%3Afalse&sort=updated
https://repo1.maven.org/maven2/com/<org>/

Find the reason someone wrote on the way out

https://raw.githubusercontent.com/<org>/<repo>/master/README.md
"this project was created when" OR "no longer actively maintained" site:github.com
"we decided to discontinue" OR "will continue internally" site:github.com

Date the real ending

https://pypi.org/pypi/<package>/json
https://registry.npmjs.org/<package>
https://hub.docker.com/v2/repositories/<org>/<image>/

Read the arguments, not the documentation

https://github.com/<org>/<repo>/pulls?q=is%3Apr+is%3Aclosed+is%3Aunmerged+sort%3Acomments-desc
https://github.com/<org>/<repo>/security/advisories?page=7
https://raw.githubusercontent.com/<org>/<repo>/master/beps/README.md
is:issue "not planned" entities performance in:title

References

Spotify, helios repository and sunset notice GitHub, archived 2024-09-12. Checked 2026-09-14.
Spotify, helios README GitHub raw. Checked 2026-09-14.
Spotify, helios tag history from May 2014 GitHub. Checked 2026-09-14.
Maven Central, com.spotify:helios-client metadata Sonatype. Checked 2026-09-14.
Spotify, apollo repository and archive notice GitHub, archived 2021-09-28. Checked 2026-09-14.
Spotify, styx repository and discontinuation notice GitHub, archived 2023-07-12. Checked 2026-09-14.
Spotify, heroic repository and deprecation notice GitHub, archived 2021-03-27. Checked 2026-09-14.
Spotify, heroic build.gradle GitHub raw. Checked 2026-09-14.
GitHub Advisory Database, GHSA-jfh8-c2jp-5v3q (CVE-2021-44228) Published 2021-12-10. Checked 2026-09-14.
Docker Hub, spotify/heroic image metadata Checked 2026-09-14.
Docker Hub, spotify/luigi image metadata Checked 2026-09-14.
Spotify, docker-client repository and notice GitHub, archived 2022-03-21. Checked 2026-09-14.
Spotify, dockerfile-maven repository GitHub, archived 2022-03-31. Checked 2026-09-14.
Spotify, spydra repository GitHub, archived 2022-03-31. Checked 2026-09-14.
Spotify, zoltar repository GitHub, archived 2023-10-18. Checked 2026-09-14.
Spotify, archived repository listing GitHub. Checked 2026-09-14.
Spotify, archived repository listing, page 2 GitHub. Checked 2026-09-14.
Spotify, active repository listing GitHub. Checked 2026-09-14.
Spotify, flink-on-k8s-operator GitHub. Checked 2026-09-14.
Spotify, dbeam repository GitHub. Checked 2026-09-14.
Spotify, luigi repository GitHub. Checked 2026-09-14.
PyPI, luigi release history Checked 2026-09-14.
PyPI, annoy release history Checked 2026-09-14.
PyPI, voyager release history Checked 2026-09-14.
Spotify, voyager repository GitHub. Checked 2026-09-14.
Tereshko, T., Spotify's journey to the cloud: why Spotify migrated its event delivery system from Kafka to Google Cloud Pub/Sub Google Cloud blog, 2016-03-03. Checked 2026-09-14.
Tereshko, T., Spotify's experiments with stream processing on Google Cloud Dataflow Google Cloud blog, 2016-03-11. Checked 2026-09-14.
Maven Central, com.spotify artifact directory Sonatype. Checked 2026-09-14.
Backstage, project repository CNCF. Checked 2026-09-14.
Backstage, ADR001: Architecture Decision Record log GitHub raw. Checked 2026-09-14.
Backstage, ADR002: Default Software Catalog File Format GitHub raw. Checked 2026-09-14.
Backstage, Enhancement Proposal process GitHub raw. Checked 2026-09-14.
Backstage, BEP 0003: Auth Architecture Evolution Created 2024-01-28. Checked 2026-09-14.
Backstage, BEP 0002: Dynamic Frontend Plugins Created 2024-01-17, provisional. Checked 2026-09-14.
CNCF Technical Oversight Committee, Backstage sandbox, incubation and governance issues GitHub. Checked 2026-09-14.
Backstage, published security advisories GitHub. Checked 2026-09-14.
Backstage, advisory listing page 3 GitHub. Checked 2026-09-14.
Backstage, advisory listing page 6 GitHub. Checked 2026-09-14.
Backstage, advisory listing page 7 GitHub. Checked 2026-09-14.
Backstage, GHSA-g2v8-7jhw-pp8p Published 2026-08-28. Checked 2026-09-14.
Backstage, issue 26665: fetch timeouts with 14k user entities GitHub. Checked 2026-09-14.
Backstage, issue 27907: slow facet queries GitHub. Checked 2026-09-14.
Backstage, PR 20409: continued MySQL support, closed unmerged GitHub. Checked 2026-09-14.
Backstage, PR 32623: queue service, closed unmerged GitHub. Checked 2026-09-14.
Backstage, closed unmerged pull requests by discussion volume GitHub. Checked 2026-09-14.
Backstage, community-plugins repository GitHub. Checked 2026-09-14.
Red Hat, Red Hat Developer Hub GitHub. Checked 2026-09-14.
Backstage, ADOPTERS.md GitHub raw. Checked 2026-09-14.
Backstage, release listing GitHub. Checked 2026-09-14.
npm registry, @backstage/backend-common Checked 2026-09-14.
npm registry, @backstage/plugin-scaffolder-backend Checked 2026-09-14.
npm registry, @backstage/core Checked 2026-09-14.
Backstage, frontend system documentation GitHub raw. Checked 2026-09-14.
Backstage, backend system documentation GitHub raw. Checked 2026-09-14.
Backstage, 2026 roadmap issue Opened 2025-12-09. Checked 2026-09-14.
Spotify, XCMetrics GitHub. Checked 2026-09-14.
crates.io, confidence_resolver crate metadata Checked 2026-09-14.
Spotify, scio GitHub. Checked 2026-09-14.