advanced 2 min answer

A location pipeline must choose a partition key. What are the candidates, and what does each optimise for?

olapartitioningorderingskewhot-key
Show the full answer Hide the answer

The candidates

  • By vehicle or driver. Gives per-vehicle ordering, which is what the state machine needs, and distributes evenly since vehicles are numerous and roughly comparable. Usually correct.
  • By geographic cell. Gives locality for spatial queries and produces severe skew — a dense city centre cell generates orders of magnitude more traffic than a suburban one, and the consumer for that partition becomes the bottleneck while others idle.
  • By city or region. Coarse, heavily skewed, and useful only if downstream processing is genuinely regional.
  • Random or round-robin. Perfect distribution and no ordering guarantee at all, which breaks any per-entity state transition.

The principle

Choose the key so that each invariant requiring ordering is contained within one partition. For vehicle telemetry the invariant is per vehicle — positions and state transitions for one vehicle must be applied in order — so the vehicle is the key.

Ordering demanded more broadly than the invariant requires is a scalability ceiling bought for nothing.

What the choice costs later

  • Partition count is expensive to change. Increasing it rehashes keys, so one vehicle's events land in two partitions during the transition and ordering breaks exactly when you are least prepared. Over-provision up front.
  • Hot keys break the model. A key generating far more than one partition can absorb cannot be split without losing its ordering — the options being a dedicated topic, a composite key where per-key ordering is not genuinely needed, or accepting the ceiling.
  • The consumer's concurrency model must preserve what the transport gave. A consumer handing partition events to a thread pool has discarded the ordering it paid for — the most common way it breaks, and invisible in testing because it only manifests under load.

The spatial requirement that tempts the wrong key

Downstream consumers frequently want geographic locality — "all vehicles in this area" — which argues for a geographic key and produces the skew.

The correct resolution is to partition by vehicle and build a geographic index downstream, rather than distorting the transport's partitioning to serve a query pattern. The transport's key serves ordering; the serving layer serves queries, and conflating them produces a pipeline that is bad at both.