advanced 2 min answer

A messaging platform stores messages partitioned by channel identifier. Some channels are enormously more active than others. Fix the partitioning.

discordpartitioningskewcompound-key
Show the full answer Hide the answer

What the interviewer is testing

Whether you reach a compound key and can state precisely what it costs.

The problem

Channel activity is enormously skewed. Most channels are quiet; a small number are extremely active, and a single large community's main channel can generate more traffic than thousands of ordinary ones combined.

A partition is served by specific nodes, so a hot channel concentrates read and write load on those nodes regardless of cluster capacity. Adding nodes does not help — the partition does not split.

The fix

Compound the key with a time bucket: partition by channel_id combined with a bounded time window. A partition then holds one channel's messages for a period, so even the busiest channel's data spreads across many partitions over time.

This is what Discord publicly describes, and the reason it is the right compound dimension rather than a random suffix is that it preserves the access pattern. Messages are read most recently first, so a typical read touches one or two buckets. A random suffix would distribute equally well and destroy the locality that makes reads cheap.

The costs

Reading long history spans multiple partitions, so the client iterates buckets — more complex, and usually acceptable because deep history reads are rare.

Bucket size is a tuning decision. Too large and hot partitions return; too small and ordinary reads fan out across many partitions. It should be derived from the busiest channel's message rate, not chosen.

Migration is a project. Changing a partition key means rewriting the data, which is why this is a one-way door worth getting right early.

What a strong answer adds

The design-time discipline: model the key distribution with real data before choosing, and assume skew. Every real-world distribution is skewed, and a key chosen from the logical model rather than the measured distribution produces a hot partition eventually.

And the general remedy shape: add a dimension along which the hot entity spreads — time, a bounded random suffix, or a sub-entity — choosing the one that preserves the property you actually need, which is usually locality or ordering.

Common weak answers

More partitions, which does not split the hot one. A random suffix, which distributes and destroys read locality.