An internal API is fast in isolation but a page that uses it takes 2 seconds for users in Asia while taking 200 ms in the origin region. The backend is equally fast for both. Diagnose and fix.

What is being tested

Whether you distinguish latency from bandwidth and recognise that round-trip count is the variable you control.

The mechanism

Latency is bounded below by the speed of light in fibre. A round trip between Asia and a European or US region is 150–250 ms, and no amount of money or server tuning changes it.

If the page makes eight sequential calls, that is eight round trips — 1.6 seconds of pure waiting before any server-side time is counted. In the origin region the same eight round trips cost 8 ms total and are invisible, which is exactly why the problem was not caught in development.

Add TCP handshake and TLS negotiation for any new connection (two to three further round trips), and TCP slow start meaning short transfers never reach full speed, and the gap widens further.

The fixes, in order of effectiveness

1. Reduce round trips. This is the only lever that helps without moving infrastructure.

• Parallelise what is currently sequential. Eight parallel calls cost one round trip, not eight.
• Batch. One endpoint returning what the page needs beats eight endpoints returning pieces of it. This is a strong argument for a backend-for-frontend that aggregates close to the services.
• Eliminate. Some calls are avoidable with denormalisation or a precomputed view.

2. Shorten the distance. Serve from a location near the user: a CDN for static assets, an edge function for logic that can run there, or a regional replica for read-heavy data. This is the only lever that reduces the cost per round trip.

3. Connection reuse. Keep-alive and connection pooling avoid paying handshake and slow start repeatedly. A client that opens a new connection per request is slow in a way no server tuning fixes.

4. Smaller payloads. Helps most where bandwidth is also constrained, which correlates with distance.

The diagnostic habit worth having

A latency budget, stated and monitored. "This page has 800 ms; the network is 200 of it; each backend call gets 60." Without one, a new dependency that adds a round trip is invisible until users in the furthest region complain — which is a slow, expensive feedback loop.

And measure where the user is. p99 measured at the server excludes DNS, TLS, the client's network and rendering — most of what the user experiences. Real user monitoring segmented by region turns this class of problem from a mystery into a number.

The general principle

You cannot optimise your way out of a latency problem by adding capacity. Make fewer round trips, or make them shorter. Everything else is secondary.