advanced 2 min answer

A trading platform faces its largest load at a precisely known time every day. What does readiness for a scheduled peak involve beyond capacity?

zerodhapeakreadinesswarmuprehearsal
Show the full answer Hide the answer

Beyond capacity

Instance count is the easy part. The failures at a scheduled peak are usually elsewhere:

  • Cold caches. A fleet scaled up ten minutes before the open has empty caches, so the first minute sends every request to the database. Cache warming is a required pre-peak step, not an optimisation, and it must be verified rather than assumed.
  • Cold connection pools. Establishing thousands of connections simultaneously at the open is itself a load spike against the database, and TLS handshakes at that volume are meaningful CPU. Pools should be pre-warmed and held.
  • Just-in-time compilation and runtime warm-up. A freshly started process is measurably slower for its first few thousand requests, which is precisely the window that matters.
  • Downstream ceilings. The exchange connection limit, the market-data feed's capacity, the notification provider's quota, the identity provider's throughput. The peak finds whichever is lowest, and they are frequently not owned by the team doing the planning.
  • Scheduled work that must not run. Backups, index rebuilds, batch jobs and deployments landing in the peak window are a recurring self-inflicted cause.

The readiness checklist that actually works

  • A change freeze covering the peak window, with an explicit exception process.
  • Pre-scaling on a schedule, not on a metric — the ramp is faster than any autoscaler.
  • A rehearsed shedding policy, with priorities decided in advance and the path exercised in production at least once.
  • A pre-peak verification that capacity, cache state, pool state and dependency headroom are actually as planned, rather than as configured.
  • A named person watching, with authority to shed, disable features and roll back without seeking approval.

The part that improves over time

Compare forecast against actual after every peak, and record it. The distribution of forecast error is the single most valuable artefact a team with recurring peaks can build, and without it the same shortfall repeats because each event is treated as new.

The property that distinguishes mature operations

Nothing about the peak is discovered on the day. The capacity is provisioned, the freeze is in effect, the shedding path has been run, the dependency limits are confirmed, and the person watching has done it before. Every one of those is a process rather than an architecture, which is why peak readiness is frequently the gap in technically excellent systems.