Service Mesh Networking

Definition

A mesh intercepts service-to-service traffic in a proxy — historically a sidecar container in every pod — and applies discovery, load balancing, retries, timeouts, circuit breaking, mutual TLS and telemetry, configured centrally.

The network-level reality

Every call traverses two proxies: the caller's and the callee's. That is two extra hops of userspace processing, typically a small number of milliseconds, plus CPU and memory per sidecar multiplied by pod count. At large pod counts the resource overhead alone becomes a meaningful fraction of cluster capacity.

The properties this buys are genuinely hard to obtain otherwise:

• mTLS everywhere with automatic certificate rotation. Doing this in application code across several languages is a large, ongoing programme.
• Consistent golden metrics for every service, with no team instrumenting anything.
• Uniform resilience policy upgradeable without redeploying applications — which is the strongest argument, because it fixes the problem that resilience libraries must be reimplemented per language and upgraded in lockstep.

The architectural shift worth knowing

Sidecar-per-pod is expensive at density, so the direction of travel is toward models that separate concerns: a shared per-node proxy handling the cheap, universal work (mTLS, L4 telemetry, identity), with an optional per-workload proxy only where L7 features are actually needed.

The reasoning generalises beyond meshes: when a per-instance cost is multiplied by a large instance count, moving the common part of the work to a shared component is usually the next architecture. The same logic produced connection poolers, shared caches and node-level log agents.

Failure scenarios

• Retries at multiple layers. Mesh retries plus client library retries plus gateway retries is exponential amplification precisely when the downstream is struggling. This is the single most common way a mesh causes an outage.
• Control plane unavailability breaking the data plane, when it should continue on last-known configuration. Verify rather than assume.
• Sidecar startup ordering, where the application starts before the proxy is ready and its first calls fail. A recurring and unglamorous source of deployment flakiness.
• Latency-sensitive paths where two proxy traversals are a material fraction of the budget.
• Adopted for observability alone, which is usually obtainable far more cheaply.

Trade-offs

Bought: uniform security and resilience policy, language independence, consistent telemetry, and independent upgrade of connection logic. Sold: per-hop latency, per-pod resources, significant operational complexity, and a subtle new failure surface that application teams must learn to recognise.

Interview question

"How would you decide whether a mesh's latency cost is worth paying for a given service?"