A platform must add a mandatory authentication header and per-caller rate limiting to service-to-service traffic. There are 180 services in 6 languages across 2400 pods, the change has to reach every service within two weeks, and the latency budget for an internal hop is 15 ms. Where should that code run?

The deciding property

Not latency, and not elegance. The deciding property is who can trigger the change. A two-week deadline across 180 services means the platform must be able to roll the change itself. Any placement that requires 180 teams to merge, build and release puts the date in the hands of 180 backlogs.

The second property is direction. This is east-west traffic between services, so anything that only sees north-south traffic cannot enforce it at all.

Why the sidecar

A sidecar is a separate process in the pod that the platform injects and owns. The platform updates the proxy image and restarts pods on its own schedule, so fleet coverage is a rollout the platform controls rather than a campaign it runs. Two weeks for 2,400 pods is a normal rolling restart.

It is also language-blind. The policy is implemented once, not six times, which matters because six implementations of the same rate limiter will not agree - they will differ in clock handling, in what counts as a request, and in behaviour at the limit, and those differences surface as an incident nobody can reproduce.

Why the other options fail

• A versioned client library is the right answer when there are one or two languages. Here it means six implementations and 180 release trains. Even with automated pull requests, the last 20 services take months, and until then the policy is not a policy.
• A shared ingress gateway is cheap and the platform already owns it, which makes it tempting. It sees traffic entering the estate and is blind to service-to-service calls, so it enforces the header on the smallest part of the problem.
• Generated code in each repository is the worst of both: the generator only touches new services, and the 180 that exist already are untouched. Generated code is a copy, and a copy cannot be updated by its author.

What it costs

An Envoy-class sidecar sits around 50 to 150 MB per pod depending on how much of the cluster's configuration it is pushed, so 2,400 pods is on the order of 120 to 360 GB of memory bought for this. Each hop gains roughly half a millisecond to a millisecond on each side, so about 1 to 2 ms against a 15 ms budget. The platform also takes on a fleet-wide restart cadence, a control plane that must stay available, and a new class of incident where the proxy, not the application, is at fault. Envoy, the proxy Lyft open-sourced in 2016 for exactly this reason, moved those concerns out of applications and into a layer a platform team can operate.

When this is the wrong answer

Below roughly 30 services in one or two languages, the library wins and a mesh is overhead with a control plane attached. It is also wrong when the per-hop budget is a couple of milliseconds, because the proxy's share of that budget is no longer a rounding error, or when the policy needs application context the proxy cannot see - a tenant identifier that exists only inside the request handler has to come from a library, whatever the deadline is.