Digs

Searching X architecture returns material written to be found by that query, which means tutorials and vendor pages. The material worth having was written by an engineer explaining a real system to their peers, and it uses different words. A dig is what comes back from going after that layer on one topic: the architecture reconstructed across real systems, the decisions and the conditions that flip them, what actually broke, and numbers you can plan against. Every claim is graded, linked and dated.

90 field guides 2433 graded sources 419 postmortems 8 categories

Storage, ingestion, lakehouse, streaming pipelines, query engines and data governance.

Data Platforms 06 Sep 2026

A million clients, four hundred connections

Reconstructs the two-line connection-rationing architecture that GitLab, Notion, Figma, OpenAI, Cloudflare and Supabase all converge on, with the measured cost of idle connections, the ceilings of a single-thr…

What surprised meThe mature deployments no longer engineer connection reuse at the pool; they engineer admission control there, because the pool is the one chokepoint…

33 sources · 26 organisations · 5 postmortems 30 min read
postgresconnection-poolingpgbouncercapacityadmission-control
Data Platforms 03 Sep 2026

The disks never disappear: what "diskless" systems actually delete

Reconstructs the zero-disk pattern across nine production systems (WarpStream, Kafka's KIP-1150, Confluent Freight, Neon, turbopuffer, Datadog Husky, SlateDB, AutoMQ, Snowflake's ancestor) from their own desig…

What surprised meEvery 'diskless' system in the public record kept a small, strongly consistent, often disk-backed ordering core; Aiven's diskless Kafka removed the b…

24 sources · 20 organisations · 4 postmortems 30 min read
object-storages3kafkadurabilitycost
Data Platforms 29 Aug 2026

The cutover is the easy part

A field guide to online resharding, reconstructed from thirteen production accounts (Notion, Figma, Slack, GitHub, Shopify, Discord, Etsy, Pinterest, Stripe, Adyen, Mercari, Monzo, AWS) and six published incid…

What surprised meNo published postmortem in this corpus blames the cutover, the step every team rehearses and fears. The incidents come from the shard key chosen nine…

34 sources · 23 organisations · 5 postmortems 41 min read
shardingpostgresmysqlvitessmigration