A workload has extreme bursts lasting a few minutes, triggered by scheduled real-world events. Should the architecture use autoscaling, pre-provisioned capacity, queues, caching, admission control, serverless, or a combination?
Show the full answer Hide the answer
Why autoscaling alone fails
The ramp is near-vertical: traffic can multiply within a minute of a scheduled event. Autoscaling is a feedback loop with a response time measured in minutes — detect, decide, provision, boot, warm, register, pass health checks. By the time capacity arrives, the burst has passed and you have paid for instances you no longer need while having failed the requests that mattered.
Autoscaling is excellent for gradual and diurnal variation. It is structurally unsuited to a spike whose rise time is shorter than its own reaction time.
The combination that works
- Pre-provisioned capacity for the known peak. The event is on a schedule, so the capacity requirement is predictable. This is the primary control, and it is scheduled scaling rather than reactive scaling — provision an hour before, not when the load arrives.
- Admission control and load shedding for the unknown excess, because the forecast will occasionally be wrong and the failure mode must be bounded. Shed at the edge, before a request consumes a worker.
- Queues for anything that can be deferred — notifications, analytics, leaderboard recomputation — so the synchronous path carries only what has a deadline.
- Aggressive caching of everything read-heavy and shared, since a large share of burst traffic is many users reading the same few things.
- Reactive autoscaling as the slow backstop, for the sustained elevated load after the spike rather than for the spike itself.
What serverless changes and does not change
Serverless removes the provisioning delay for the compute tier, which genuinely helps. It does not remove the downstream constraint: ten thousand concurrent functions all opening database connections is a worse problem than a queue. Serverless in front of a stateful backend moves the bottleneck rather than removing it, unless a connection proxy or a queue sits between them.
It also has its own cold-start behaviour, which for a spike arriving in seconds is the same category of problem autoscaling has, in a smaller form.
The judgement being tested
Capacity planning for a predictable spike is a scheduling problem, not a scaling problem. The sophisticated-sounding answer (autoscaling, serverless) is the wrong primary control; the unglamorous answer (know the event calendar, provision ahead, shed the excess) is right. The scaling machinery is the backstop for the case where the forecast was wrong.