A mobility platform grows from a monolith to hundreds of services and finds that client-side load balancing libraries in five languages behave inconsistently during failures. What architectural move addresses this, and what does it cost?

The problem being solved

When discovery, load balancing, retries, timeouts and circuit breaking live in a library, every language gets its own implementation. In practice that means:

• Five subtly different retry policies, so a platform-wide retry-budget decision cannot be made at all.
• Inconsistent behaviour during partial failure — one library ejects unhealthy hosts, another keeps sending to them.
• Upgrading a policy requires every service in every language to adopt a new library version and redeploy, which takes quarters.
• Observability differs per language, so cross-service latency attribution is unreliable exactly when it is needed.

The deeper issue is that these are platform policies being implemented by application teams, which guarantees drift.

The architectural move

Move the concerns out of the process and into a sidecar proxy deployed alongside every service. The application makes a plain local call; the proxy handles discovery, load balancing, health checking, retries, timeouts, circuit breaking, mutual TLS and telemetry. This is the origin of the service-mesh data plane, and Envoy's development at Lyft is the canonical example.

What it buys:

• One implementation for every language, so policy is uniform by construction.
• Policy changes without application redeploys, pushed through the control plane.
• Consistent, comparable telemetry at every hop — the single most underrated benefit, because it makes latency attribution across a large service graph possible for the first time.
• Uniform mutual TLS and identity, which is otherwise a per-language security project.

What it costs

• An extra network hop each way, adding latency (typically small but not zero) and CPU per pod.
• A control plane to operate, which is a distributed system in its own right and becomes a correlated failure domain: a bad control-plane push can affect every service simultaneously.
• Significant operational learning. Mesh configuration is genuinely difficult, and debugging moves from "read the application code" to "reason about proxy configuration and the control plane".
• Resource overhead that is meaningful at large fleet sizes — a sidecar per pod is a real line item.

When it is the wrong answer

For twenty services in one or two languages, a shared library is simpler, cheaper and easier to debug. The mesh pays for itself when the number of languages and the number of teams make library consistency impossible — which is a people problem the mesh solves with technology, and it is worth being honest that this is the actual justification.

The judgement being tested

Whether you can articulate that service discovery is rarely the hard part. Discovery is a lookup. The hard parts are what you do with the result under failure, and whether every service agrees on it. That is why the answer is a policy-enforcement layer rather than a better registry.