practice

Soak Test

Running sustained realistic load for hours or days to expose defects that accumulate over time rather than appearing under peak load.

Short tests validate throughput and latency. Soak tests find the failures that need time, which are usually the ones that produce a 3 a.m. page rather than a bad benchmark.

What only a long run reveals: memory leaks, visible as a steadily rising baseline that a twenty-minute test cannot distinguish from warm-up; connection and file descriptor leaks, which manifest as sudden failure when a limit is reached; disk and log growth; cache degradation, where the working set grows past the cache and hit rate silently falls; database bloat and index fragmentation; certificate and token expiry in long-lived processes; and counter or ID overflow.

Design points: run for at least 24 hours to cover daily cycles and scheduled jobs, at realistic rather than peak load, and — critically — compare the end state to the beginning rather than only checking that it stayed up. A test that passes while memory rose 40% has found a defect and reported success.

Include deployments, restarts and failovers during the run, since production is not a steady state either.