Discord stores trillions of messages. What is their partition key, and what problem does the second half of it solve?

The key

(channel_id, bucket) — where bucket is a fixed time window.

What each half does

channel_id matches the read pattern. Clients read messages within a channel, so co-locating a channel's messages means a read touches one partition.

bucket bounds partition size. Partitioning by channel alone would put every message a busy channel has ever received into one partition — unbounded in size, permanently hot, and eventually larger than any single node should hold. Adding a time bucket caps it and matches the query pattern exactly, since clients read recent messages. The newest bucket is hot, older buckets are cold, and cold buckets can be aged out.

The rule in one line: model for the read pattern, then bound the partition. That covers most wide-column data modelling.

The problem that remained

Even bucketed, a handful of very large channels behaved differently, and concurrent requests for the same hot partition amplified load — many requests, same data, all hitting the database.

Their published fix was request coalescing in an intermediary data service: many concurrent requests for the same data produce one database query, whose result is fanned back out. This is single-flight applied at the data tier, and it is the reusable answer wherever popularity is skewed — which is nearly everywhere.

The migration history, and why it matters

MongoDB → Cassandra (2017) → ScyllaDB (2023). Each move had a specific named constraint: first the working set outgrowing memory; later JVM garbage-collection pauses causing latency spikes and compaction falling behind.

That precision is the transferable discipline. "We outgrew our database" is not a diagnosis. Naming the constraint tells you whether a different database is even the right kind of fix — and here, two of the three problems were addressed by data modelling and a caching layer rather than by the store itself.

What a strong answer adds

Noting that the Cassandra → ScyllaDB move required no data model change, because the model was compatible. The migration was an operational-characteristics decision — removing GC pauses — not a data-modelling one, and separating those two axes when evaluating a datastore is a distinction most comparisons blur.