A mobility platform must route API traffic, long-lived driver location streams and internal service calls. Which load-balancing layer suits each, and what breaks if one choice is applied uniformly?

Matching the layer to the traffic

API traffic → Layer 7. Requests are short-lived and independent, so per-request balancing spreads load well. Layer 7 provides the things this traffic needs: path and header-based routing, retries on idempotent requests, request-level observability, TLS termination, and header-based canary routing. The per-request processing cost is worth it.

Long-lived location streams → Layer 4. A driver's connection persists for hours. Layer 7's advantage — per-request decisions — is worthless when there is one logical request lasting hours, and its cost is real: buffering, header parsing and a proxy that must hold state for every connection. Layer 4 connection-level balancing forwards packets with minimal overhead and handles enormous connection counts efficiently.

Internal service calls → client-side or mesh. A central proxy on every internal hop adds a network round trip and becomes a shared failure domain. Client-side balancing — or a sidecar — lets each caller choose an endpoint directly using health and load information, removing a hop and enabling locality-aware routing that prefers same-zone endpoints (which also reduces cross-zone data transfer cost, often a surprisingly large line item).

What breaks under a uniform choice

Layer 7 everywhere. The proxy tier must hold state for millions of long-lived connections, so it becomes the memory-bound bottleneck. Connection draining during a proxy deploy disconnects every driver simultaneously — a reconnect storm caused by a routine deployment. And internal calls pay an extra hop each.

Layer 4 everywhere. API traffic loses path-based routing, so routing logic moves into the application. No request-level retries, no per-request observability, and TLS terminates in every service rather than at the edge. Canary deployments become an infrastructure problem rather than a routing rule.

DNS round-robin for all three. The classic wrong answer. No health awareness — a dead endpoint keeps receiving traffic until caches expire. TTLs are honoured inconsistently and clients cache far longer than they should. Failover time is unbounded, and the balance is per-resolver rather than per-request.

The connection-lifetime insight

The generalisable rule is that balancing granularity should match the unit of work. Short requests want request-level balancing. Long connections want connection-level balancing, and their real problem is not distribution but rebalancing: once connections are established, load imbalance persists until something forces reconnection. That needs explicit handling — connection lifetime limits with jitter, or server-initiated rebalancing — which no load-balancer choice provides on its own.