Service Mesh

Definition

A sidecar proxy runs next to each service instance and intercepts all inbound and outbound traffic. A control plane configures the proxies. The application makes a plain local call; the proxy handles discovery, load balancing, retries, timeouts, circuit breaking, mutual TLS and telemetry.

Why it matters

In a polyglot estate, resilience logic implemented as a library must be written and maintained once per language, and upgrading it requires redeploying every service. The mesh moves that logic into a component upgraded independently of application teams, which is the strongest argument for it.

It also makes two things possible that are painful otherwise: mTLS everywhere by default, and consistent golden metrics for every service without any team instrumenting anything.

When it is worth it

The honest threshold is high. A mesh earns its cost when you have many services, several languages, a real requirement for encrypted service-to-service traffic, and a platform team to own it. Below that, a library in one or two languages does the same job with a fraction of the operational surface.

Uber-scale estates and similar large polyglot organisations adopted proxy-based data planes for exactly these reasons — the library approach had become unmaintainable across languages, and consistent observability across thousands of services was otherwise unobtainable.

Failure scenarios

• Retries at multiple layers. The mesh retries, the client library retries, the gateway retries. Three layers of three attempts is 27 requests for one user action, arriving precisely when the downstream is already struggling. Decide where retries live — usually one layer — and disable them everywhere else. This is the single most common way a mesh causes an outage.
• The control plane as a single point of failure. Data planes must keep working on last-known configuration when the control plane is unavailable. Verify this rather than assuming it.
• Latency added to every hop. Two proxy traversals per call. Usually small, occasionally decisive for latency-sensitive paths.
• Debugging complexity. A connection failure may now originate in the application, the sidecar, the control-plane configuration, or the network policy. Teams need to be taught how to tell.
• Adopted for observability alone, which is often obtainable far more cheaply.

Trade-offs

Bought: uniform resilience and security policy, language independence, consistent telemetry, independent upgrade of connection logic. Sold: significant operational complexity, per-hop latency, resource overhead per pod, a new and subtle failure surface, and a platform team's ongoing attention.

The pattern to watch for: a mesh introduced to make a distributed monolith survivable. It makes the six synchronous hops observable and retryable; it does not make them fewer. Fixing the boundaries is the cheaper intervention.

Interview question

"A mesh is introduced and p99 latency rises 15% while the error rate falls. How do you decide whether that is a good trade, and what would you tune first?"