Connection Draining
also called Deregistration Delay
The period during which a load balancer stops sending new requests to an instance while allowing in-flight ones to complete before it is removed.
Without it, removing an instance — for a deployment, a scale-in, or a health-check failure — severs requests mid-flight. Users see errors during every routine deployment, which is the most common cause of the small error spike teams learn to ignore.
Draining requires cooperation from both sides, and the application half is what is usually missing:
The load balancer stops routing new requests and waits for the drain timeout before terminating.
The application must, on receiving SIGTERM, first fail its readiness check (so it leaves
rotation), then continue serving in-flight requests, then exit. Exiting immediately on SIGTERM
defeats the drain entirely — the load balancer is politely waiting for a process that has already
gone.
Two sizing details. The drain timeout must exceed the longest legitimate request, or long requests are cut anyway; and it delays every deployment and scale-in by that amount, so an unnecessarily long timeout makes rollouts slow.
The related case that gets missed: long-lived connections. WebSocket and gRPC streams do not complete on their own, so draining must include an application-level signal telling clients to reconnect, and clients must reconnect with jittered backoff or the drain produces a thundering herd.