Authentication, rate limiting and request tracing must behave identically across 60 services written in four languages, and the rules change roughly monthly. Where should that logic live?
Show the full answer Hide the answer
The deciding property
How a change reaches production, multiplied by how many runtimes must implement it. Monthly changes across four languages is the combination that settles this: a library means four implementations and 60 rebuilds per change, and a gateway means every internal call pays a detour.
This is the problem Lyft built Envoy for, open-sourcing it in 2016: it moved retries, timeouts, service discovery and observability out of polyglot application code into a process beside each service, so the behaviour could be changed without touching or rebuilding the services. That is the documented origin of the sidecar pattern in this form, and the reasoning transfers directly.
Why the sidecar
- Changes propagate by redeploying the sidecar, not by rebuilding consumers. A monthly policy change becomes a fleet rollout of one artefact, staged by cohort, with a rollback that does not involve 60 teams.
- One implementation for four languages, which is the only option here that does not multiply the work by the number of runtimes.
- It sits on the data path of the service it protects, so rate limiting and authentication apply to every call including internal ones, without a central hop.
- Uniform telemetry for free, because the proxy sees every request in a consistent form.
The costs are real: a second process per instance with its own memory and CPU footprint, roughly a millisecond of added latency per hop, a control plane that is now critical infrastructure, and a debugging experience where the proxy is in every trace. You have traded 60 rebuilds for one more thing that can be down, and the control plane must be statically stable so a control-plane outage freezes configuration rather than stopping traffic.
Why not the others
- A shared library is the right answer when the capability is stable and the estate is monoglot. Here it is the worst option: four implementations to keep in step, and every change requires 60 services to rebuild and redeploy, so the patch latency of the estate is the slowest team's release cycle. An urgent authentication fix then takes a quarter.
- A central gateway is correct at the edge, where traffic enters once and the detour is already paid. Routing internal service-to-service calls through it adds a hop each way, makes it a single point of failure for the whole estate, and scales its capacity with total internal traffic rather than with ingress. Many estates run both: a gateway at the edge, sidecars inside.
- Per-service implementations with a conformance suite is honest about variance and does not remove it. Four languages times 60 services is 60 chances to be subtly wrong about token validation, and a conformance suite tells you about the drift after someone has written it. It is the fallback when sidecars are impossible, for instance where the runtime cannot host one.
What would flip the decision
| If this changes | Choose | Because |
|---|---|---|
| The rules change yearly, not monthly | Shared library | Rebuild cost is paid rarely and you avoid a control plane |
| One language across the estate | Shared library | A single implementation removes the sidecar's main benefit |
| Traffic is nearly all north-south | Gateway | The hop is already on the path and there is nothing to duplicate |
| The estate is under about ten services | Library or per-service | A control plane and a mesh cost more than the drift they prevent |
| Per-request latency budget is a few milliseconds | Library | A millisecond per hop is a large share of the budget |
When this is the wrong answer
Adopting a full service mesh to get three cross-cutting behaviours across eight services is the classic over-build: you acquire a control plane, a new upgrade treadmill and a novel failure mode in exchange for drift you could have prevented with a code review. Below roughly ten services, or where one language covers the estate, the library wins on total cost — and the honest version of this answer names that threshold rather than recommending a mesh on principle.