A service mesh is introduced. p99 latency rises 15% and error rates fall by half. How do you decide whether that is a good trade?

What is being tested

Whether you can evaluate a trade against business impact rather than against engineering aesthetics.

The framework

1. Is the latency increase inside the budget? If the SLO is p99 under 300 ms and you moved from 180 ms to 207 ms, you spent headroom you had. If you moved from 280 ms to 322 ms, you broke the SLO and the trade is not available at that price.

The latency number in isolation means nothing. Latency matters relative to a stated target, which is why having one matters.

2. What is the error reduction worth? Halving the error rate has a value that can be estimated: failed requests, abandoned sessions, support contacts, or in a commerce context, lost transactions. Compare with the value of 27 ms.

For most systems, halving errors is worth far more than 27 ms of p99. For a small number of latency-critical paths — real-time bidding, live media, trading — it is not.

3. Is the improvement uniform or concentrated? If errors fell because of automatic retries and outlier ejection, the benefit lands on the users who were previously failing — often those on poorer networks or hitting degraded instances. That is a tail improvement paid for with a small median cost, which is usually a good trade and frequently under-valued because the beneficiaries are invisible in aggregate metrics.

What to tune first

• Check for retry amplification. If errors fell because the mesh retries, verify that client libraries and the gateway are not also retrying. Three layers of three attempts is 27 requests for one user action, arriving precisely when the downstream is struggling. This is the single most common way a mesh causes an outage, and it can look like an improvement in the metrics right up until the day it does not.
• Exempt latency-critical paths. A mesh is not all-or-nothing. Services with tight budgets can bypass it or use a lighter configuration.
• Reduce proxy work. Much of the added latency is often telemetry generation and access-log formatting rather than proxying itself. Sampling traces and trimming logged fields recovers a meaningful share.
• Check the topology. Two proxy traversals per call is inherent; more than two means something is misconfigured.

The larger question worth raising

Are you paying for the mesh's benefits, or working around a structural problem? If the error reduction comes from retrying calls in a chain of six synchronous hops, the mesh has made a distributed monolith survivable rather than fixing it. Fewer hops would give you both lower latency and fewer errors, and it is the cheaper intervention.

The direction of travel worth knowing

Sidecar-per-pod is expensive at density — two proxy traversals plus CPU and memory per pod multiplied by pod count. The industry direction is separating concerns: a shared per-node proxy for the cheap universal work (mTLS, L4 telemetry, identity) with a per-workload proxy only where L7 features are genuinely needed. The general principle is worth recognising: when a per-instance cost is multiplied by a large instance count, moving the common part to a shared component is usually the next architecture — the same logic that produced connection poolers, shared caches and node-level log agents.