advanced 3 min answer

A flash sale opens at a published time. The precomputed inventory and price data is served from a real-time serving layer whose caches were rebuilt an hour earlier during a deploy. What happens in the first ten seconds after the sale opens?

flash salecold cacherequest collapsingalibabapeak readiness
Show the full answer Hide the answer

Second by second

At second zero, arrival rate goes from baseline to peak in under a second, because the start time was published and every client is waiting on it. Alibaba's Singles' Day peak of 583,000 order creations per second in 2020 is the documented upper end of what this shape of traffic looks like; the mechanism is the same at a thousandth of that scale.

  • Seconds 0 to 2: every request misses the serving layer's cache for the hot items. Each miss becomes a read of the underlying store. Because thousands of requests miss the same key simultaneously, the store receives thousands of identical reads for one row, which is the cache stampede.
  • Seconds 2 to 5: the store's latency rises under the duplicated load. Serving-layer requests that were waiting now time out. Timeouts convert into retries, and retries arrive faster than the original requests did, so offered load increases while goodput falls.
  • Seconds 5 to 10: connection pools saturate. The failure is now visible on endpoints unrelated to the sale, because they share the pool and the store.

Where it amplifies

The amplifier is not the traffic, it is the duplication. One cold key under 50,000 concurrent requests is 50,000 reads that should have been one. The second amplifier is retry behaviour: a client library retrying three times with a short backoff turns one spike into four.

What the user sees

For the first seconds, a slow page rather than an error, which is worse: users refresh, adding load, and the refresh arrives without the cache headers that would have let it be served stale.

What stops it

  • Request collapsing (single-flight) at the serving layer, so concurrent misses for one key produce one origin read and share the result. This is the single highest-value mechanism for this failure and it is a local change.
  • Warming the specific keys before opening, driven by the known sale catalogue. The sale items are a list somebody has; loading them is a script, not an architecture.
  • Serving stale on miss while refreshing asynchronously, which turns a correctness question into a product question: an inventory count two seconds old is almost always acceptable, and the reservation step is where accuracy is actually required.
  • A retry budget rather than fixed retries, capping retried requests at a small fraction of first attempts so the client cannot multiply an incident.

What would have to be true for it to self-heal

Only that the miss storm is bounded: with collapsing, the first requests warm the keys and everything after second one is a hit. Without collapsing, it does not self-heal, because each arriving request finds the key still unpopulated and joins the storm. That is the distinction worth remembering: cold caches recover on their own only when concurrency is low enough that the first fill completes before the next wave arrives.

When this is over-engineering

A sale with a few hundred concurrent buyers has no stampede: the origin serves a few hundred duplicate reads and nobody notices. The mechanisms above start paying at the point where concurrent misses for a single key exceed what the origin serves comfortably in one round trip, which for most relational stores is in the low thousands. Below that, warming the catalogue with a script is the whole answer.