TCP/IP for Architects

Definition

TCP provides an ordered, reliable, congestion-controlled byte stream. UDP provides none of that and gets out of the way. The architectural consequences of that difference are larger than most designs account for.

The four things that matter architecturally

Connection setup is not free. A TCP handshake is one round trip; TLS adds one or two more. Over a 150 ms intercontinental path, that is 300–450 ms before any application byte moves. This is why connection reuse, keep-alive and connection pooling matter more than almost any application-level optimisation, and why a client that opens a new connection per request is slow in a way no server tuning will fix.

Slow start means short connections never reach full speed. TCP begins conservatively and ramps. A connection transferring 50 KB and closing spends its whole life in slow start. Long-lived pooled connections avoid paying this repeatedly.

Head-of-line blocking. TCP guarantees ordered delivery, so one lost packet stalls everything behind it until it is retransmitted — even data that arrived fine and belongs to an unrelated multiplexed stream. This is the fundamental limitation that HTTP/2 over TCP could not solve and that QUIC addresses by moving reliability above the transport.

Congestion control is a shared negotiation. Throughput on a lossy path is bounded by the algorithm's reaction to loss. On networks where loss is due to interference rather than congestion — mobile, satellite — classic loss-based algorithms badly under-use available capacity.

Industry example

Real-time communication is where TCP is the wrong choice, and understanding why is more instructive than the rule. In a Zoom-style media path, a packet of audio that arrives 400 ms late is worthless — the conversation has moved on. TCP would retransmit it, delaying everything behind it to deliver data nobody wants.

So media runs over UDP, and the application implements exactly the reliability it needs: forward error correction so some loss is recoverable without retransmission, a jitter buffer trading a little latency for smoothness, and codecs that degrade gracefully when packets are missing.

The architectural principle generalises: reliability should be implemented at the layer that knows what the data is worth. TCP's guarantee is unconditional and therefore sometimes wrong.

Failure scenarios

• A new connection per request, paying handshake and slow start every time.
• Undersized connection pools, so requests queue for a connection while the network is idle.
• Idle timeouts mismatched between client, load balancer and server, producing connection resets that look like application errors.
• Nagle's algorithm interacting with delayed acknowledgement, adding tens of milliseconds to small request/response exchanges — the classic cause of a mysterious 40 ms floor.
• A firewall silently dropping idle connections, so a pooled connection is dead but appears open until first use.

Interview question

"Why does a chatty API over an intercontinental link perform badly even when both endpoints are fast, and what would you change?"