A retail platform's load tests pass comfortably and production fails at the same throughput during a sale. What is different about production?

The differences that matter

• Data shape. Synthetic tests usually generate uniformly distributed keys. Real traffic is extremely skewed — a small number of products, a small number of categories, a small number of sellers. Uniform tests produce an unrealistically good cache hit rate and never create the hot-key contention that actually breaks things.
• Cache state. A load test that runs for twenty minutes against a warm cache is testing a different system from a sale opening against caches that have just been invalidated by a price update.
• Data volume. Tests against a small dataset have every index in memory and no realistic query plans. A table with a hundred million rows behaves differently, and the difference appears as a plan change rather than a gradual slowdown.
• Concurrency mix. Real traffic is browse-heavy with a small share of writes; tests frequently over-weight the write path because that is what the team was worried about, and thereby miss the read-path contention that dominates.
• Third-party behaviour. Tests use mocks that respond in a millisecond. Real payment providers slow down during a sale because everyone else's sale is also happening.
• Duration. A twenty-minute test does not surface memory leaks, connection leaks, or the effects of log and metric accumulation that appear after hours.

What to change

• Replay or shape traffic from production, including the popularity distribution. This single change finds more real problems than any other.
• Start with cold caches, at least in one scenario, since that is the state during the event that matters.
• Test against production-scale data, or accept that query plans are untested.
• Inject dependency latency during the test, because a load test with healthy dependencies validates a scenario that will not occur.
• Run a soak test for hours to find the slow leaks, separately from the burst test.
• Push past the target until it breaks, since the interesting number is where throughput stops increasing and most load tests stop at the target and never find it.

The conclusion that follows

A load test that passes tells you very little; a load test that finds the breaking point tells you a lot. The purpose is to locate the constraint and the failure mode, not to confirm a number — and a team that has never seen its system fail in a test will see it fail for the first time in production.