Network Performance Tuning

Definition

Network performance work is dominated by round trips and connection handling, not by bandwidth or kernel parameters — which is the opposite of where most tuning effort goes.

In order of effect

1. Reduce round trips. Parallelise sequential calls, batch, denormalise, precompute. The only lever that helps without changing infrastructure, and usually the largest.

2. Reuse connections. Keep-alive and connection pooling avoid paying handshake plus TLS negotiation (two to three round trips) and TCP slow start on every request. A client opening a new connection per request is slow in a way no server tuning fixes.

3. Align timeouts across the path. Client, load balancer, proxy and server idle timeouts that disagree produce connection resets that present as random application errors. The server's idle timeout should exceed the load balancer's, or the balancer will reuse a connection the server just closed.

4. Compress, selectively. Large text payloads benefit greatly. Compressing already-compressed content — images, video — wastes CPU. Small payloads may cost more in CPU than they save in bytes.

5. Move the data closer. CDN, edge, regional replica. Reduces the cost per round trip, which nothing else does.

6. Then kernel and socket tuning. Buffer sizes, congestion control algorithm, ephemeral port range, connection backlog. Real but usually a smaller effect than the above, and easy to get wrong.

The specific traps

• Nagle's algorithm interacting with delayed acknowledgement, adding tens of milliseconds to small request/response exchanges. The classic cause of a mysterious 40 ms floor.
• Ephemeral port exhaustion on a host making many outbound connections, presenting as intermittent connection failures.
• DNS resolution on every request, or a runtime caching resolution forever and ignoring TTL.
• Cross-availability-zone traffic on every request, adding both latency and transfer charges.
• TLS renegotiation or session resumption not configured, paying full handshake cost repeatedly.

Measuring properly

Measure at the client, not the server. Server-side timing excludes DNS, connection setup, TLS, transfer and rendering — which is most of what the user experiences. Real user monitoring segmented by region and network type turns a mystery into a number.

Interview question

"An internal API call takes 40 ms consistently, and the server reports 2 ms of processing. Where is the time going?"