Partition Assignment
The mapping of partitions to consumer instances that determines parallelism, ordering guarantees and what happens when the consumer set changes.
Competing consumers scale processing by adding instances. The constraint is that ordering is guaranteed only within a partition, so a message's partition key determines both which instance handles it and what order it is seen in.
Three consequences that shape design:
Parallelism is capped by partition count. Twenty consumers on ten partitions leaves ten idle. Partition count is therefore a capacity decision made early and awkward to change later, because increasing it changes the key-to-partition mapping.
Key choice creates hot partitions. Partitioning by a value with skewed distribution — a large tenant, a popular product — puts most traffic on one partition and one consumer, and no amount of scaling helps.
Rebalancing is disruptive. When an instance joins or leaves, partitions are reassigned; processing pauses, and in-flight work may be reprocessed. Frequent rebalancing — usually caused by consumers failing heartbeats while doing long work — can consume most of the throughput. Cooperative or incremental rebalancing reduces the impact.
The related failure is the poison message: one item that cannot be processed blocks its partition entirely. A dead-letter path with a retry limit is mandatory, not optional.