Data Platforms 20 Sep 2026 35 min read

When the database cannot be split: ten years of GitLab.com, read from its own incidents and design documents

How GitLab's stateful tier changed between 2016 and 2026, reconstructed from the company's own public primary record: a decade of production incident issues, the architecture design documents including the ones marked rejected, a sharding blueprint closed without merging, and a service catalogue that can be counted rather than estimated.

A single-company archaeology of the problem every growing platform eventually hits: two components that cannot be sharded without rewriting the application above them. A reader leaves with a decision tree for saturating state that has no horizontal axis, a failure catalogue in which the two classes that span the whole decade are undetected state divergence and mitigations that become the next incident, a size threshold at which a table stops being a query problem and becomes a release-engineering problem, and a set of API queries that turn any public GitLab instance into an incident corpus.

The finding that surprised me

The decade's most consequential number is a table size, not a latency: index builds on a 1.5 TB table ran 1.5 to 6 hours and blocked deployment, so GitLab set a 100 GB target and enforced it with static analysis that refuses new columns, converting a capacity limit into a lint rule.

What you get out of it

  • Two of the three attempts to scale Git storage horizontally were built and then removed: Praefect's read distribution 'has basically failed', and the Raft and write-ahead-log transaction manager is being unwound in a merge request of 365 files that is too large to review at once.
  • The size at which a table becomes a release-engineering problem is roughly an order of magnitude below the size at which it becomes a query-performance problem, and you hit the first one first.
  • The corrective action from the January 2017 data-loss incident had silently stopped existing by August 2018, because the backup cron did not survive the move to Google Cloud and nothing alerted on its absence.
  • GitLab's own incident review records that the fix for the February 2026 outage could not be merged, because merging it required the CI system the outage had disabled.
  • A mitigation that defers work into the resource it is protecting is an amplifier: deferred audit-event jobs exhausted Redis memory, broke quorum, produced a 12-million-job backlog, and recurred a week later before the corrective actions shipped.

Scope

Why this, now. Two GitLab design documents written in 2026 name AI agents as the new load and structure driver, one because agentic traffic is raising per-repository Git load and one because agents cannot carry the implicit context a monolith assumes, which makes this the moment the decade's constraints get re-argued on new grounds.

What it does not cover. The product, the Duo model stack beyond where it appears in incidents, security and compliance architecture, the self-managed and Dedicated packaging paths, and all cost: this session's network policy resolved gitlab.com and refused every other host, including GitLab's blog, documentation and status page, so the corpus is one host and every performance figure in it is GitLab measuring GitLab.

Open the field guide → Self-contained: it loads nothing at read time, follows your system theme, and prints cleanly.