A global SaaS product has a 200 ms p95 API budget. Users in Sydney see 900 ms against a single US-East deployment. The application team says the service responds in 40 ms. Who is right and what do you do?
Show the full answer Hide the answer
Both are right, and that is the point
The service does respond in 40 ms. The other 860 ms is network — and no amount of application profiling will show it, which is why these arguments stall.
Do the arithmetic before proposing anything
Sydney to US-East is roughly 200 ms round trip. Now count round trips on a cold request:
DNS resolution (1, more if there is a CNAME chain), TCP handshake (1), TLS handshake (1–2), then the request itself (1). That is five round trips before the 40 ms of work — a full second, and it matches the reported number.
This calculation takes two minutes and should precede any optimisation discussion.
The two levers, in order of return
Remove round trips. Cheapest and available immediately.
- Connection reuse — keep-alive and pooling remove the handshakes for every request after the first. For a chatty client this is the single largest win.
- HTTP/2 or HTTP/3 — multiplexing removes serialisation across requests; QUIC folds TLS into the transport handshake and gives 0-RTT on resumption.
- Remove CNAME chains in DNS, and batch or parallelise dependent client calls.
Move closer. Larger ceiling, larger cost.
- CDN with TLS termination at the edge — the handshakes now happen against a point of presence near Sydney and only the request traverses the long path. This is often overlooked because the API is not cacheable; termination alone is worth it.
- Read replicas or a regional deployment — the real fix, and a data-architecture decision, not a networking one. It brings replication lag, write routing and residency into scope.
- Edge compute for auth and validation, so rejections never cross the ocean.
The framing to bring back to the business
Speed of light is a constraint, not a bug. The 200 ms budget is achievable in Sydney only with regional infrastructure. So the honest options are: fund regional presence, or set regional SLOs.
Pretending a single-region deployment can meet a global latency target is how teams spend quarters optimising code that was never the problem.