Zero-Downtime Deployment in Practice

Definition

Deploying a new version while continuing to serve traffic, with no interruption visible to users.

Why it matters

The obvious reason is availability. The more important one is batch size: if deploying requires a maintenance window, deployments are rare, so each contains many changes, so each is riskier and harder to diagnose. Zero-downtime deployment is what makes small frequent changes possible, and small frequent changes are what reduce change failure rate.

The governing constraint

During any rolling deployment, two versions of the code are live simultaneously. Everything else follows from this.

It means every change must be compatible with the version it is replacing — in both directions, since a rollback runs the old code against data the new code wrote.

Implementation patterns

Expand and contract for any schema or contract change. Expand: add the new column or field, nullable and unused; deploy. Migrate: write to both, backfill in batches, move reads. Contract: remove the old, only after telemetry — not a code search — confirms nothing reads it. The contract step is a separate deployment, days or weeks later.

Backward-compatible API and event changes: add optional fields, never change a type, never repurpose a field's meaning.

Tolerant readers, ignoring unknown fields so a producer can add without coordinating.

Connection draining, so an instance being removed finishes in-flight requests rather than cutting them. The application must handle the termination signal, stop accepting work, finish what it has, and exit — which many runtimes do not do by default.

Readiness gating, so an instance receives traffic only when it can serve it.

Feature flags to separate deployment from release, so the code ships dark and the behaviour is enabled independently.

Failure scenarios

A destructive migration bundled with application code, which removes rollback permanently — the old version cannot find the column it needs, so recovery must be a forward fix under pressure.

Long-running requests or WebSocket connections cut at cutover because the drain timeout is shorter than the longest request, or the platform's grace period is shorter than the drain timeout.

Background workers and message consumers overlooked entirely, because they sit outside the load balancer and need their own shutdown handling — finish the current message, commit the offset, stop polling.

Cache and JIT cold start, where new instances take traffic before they are warm and latency spikes.

Industry example

The pattern is what makes continuous deployment at Etsy-like frequency possible: many deployments a day, each small, none requiring a window. The organisational consequence is the one worth noting — frequent deployment is not a tooling achievement, it is the result of every schema and contract change being made compatible by discipline.

Trade-offs

Expand-and-contract is three deployments where one would do, and the contract step is boring and routinely never done — which is how systems accumulate columns with version numbers in their names that three services still dual-write to.

The alternative, a maintenance window, is simpler per change and produces larger batches, rarer deployments and worse outcomes.

Interview question

You deploy at 14:00. At 14:20 errors spike. The release included a migration that dropped a column. What do you do?

The candidate must recognise that rollback is unavailable — the previous version needs that column — and choose forward fix, mitigating first with a flag or by shedding the failing path. Strong answers add the pipeline control: fail any migration containing a destructive statement unless it carries an explicit approval marker.