Lyft open-sourced Envoy in September 2016 after running it internally, moving retries, timeouts, service discovery and network telemetry out of its services and into a sidecar proxy. Which problem did centralising the network layer solve that a shared client library could not, and what did the sidecar cost?

The situation they were in

A service estate in several languages, where every service implemented its own retries, timeouts, load balancing and metrics — each slightly differently. The stated goal in the 2016 announcement was that the network should be transparent to applications, and when something goes wrong it should be obvious where. In practice the opposite held: when a call failed, the first twenty minutes of every incident went on establishing whose timeout fired and whether a retry had amplified the problem.

What they chose

A process-local proxy, written in C++, that every service talks to over localhost. All outbound and inbound traffic goes through it, so retry policy, timeouts, circuit breaking, service discovery and consistent telemetry are configuration rather than code, and they are identical for every service regardless of language.

Envoy was announced publicly in September 2016, joined the CNCF in 2017 and graduated in 2018, which matters here only as evidence that the model held up outside the organisation that built it.

Why a library was not enough

A library's rollout speed is bounded by the slowest team that must adopt it. Changing a retry policy fleet-wide means a version bump, a rebuild and a redeploy of every service, in every language, coordinated across teams — months, and never complete. A sidecar changes the same policy by pushing configuration to running processes.

The second reason is uniformity of measurement. Metrics emitted by the proxy are directly comparable across services because the same code emits them, whereas per-language libraries disagree about what a timeout is and where latency is measured. Consistent measurement is what makes "which hop is slow" answerable at all.

What it cost

• Two extra hops per call — client sidecar and server sidecar — costing sub-millisecond each in normal operation, which matters for a chatty internal path making dozens of calls per request.
• Memory and CPU per instance, on the order of tens of megabytes of resident memory per sidecar, multiplied by every pod in the fleet.
• A new fleet-wide blast radius. The configuration plane now reaches every service, so a bad push is a global event rather than one team's outage. This is the central trade of the whole pattern: the thing that makes policy changes fast makes mistakes fast too.
• Specialised expertise. Someone must understand the proxy deeply, and during incidents that person is on the critical path.

When copying this would be the wrong answer

A single-language estate should use a library. Most of Envoy's value came from polyglot services, and a team running only Go or only Java gets the same policy uniformity from one dependency without the hops, the memory or the control-plane blast radius. Below roughly a dozen services with one deployment team, the operational cost of a mesh exceeds the coordination cost it removes. The condition that flips it is the second language, or the point where changing a timeout fleet-wide takes longer than an afternoon.