intermediate 2 min answer

Your system handles 1,000 requests per second today. Marketing says a campaign will bring 10,000 next month. What breaks first, and how do you find out?

capacitybottleneckload-testingscaling
Show the full answer Hide the answer

What the interviewer is testing

Whether you reason about bottlenecks systematically or start adding servers.

The shape of the answer

Scaling is not uniform. Some components scale linearly with an autoscaler, some hit a hard ceiling, and the useful skill is knowing which is which before the traffic arrives.

Almost certainly fine: stateless application servers behind a load balancer, the CDN, object storage. These scale horizontally and the cloud provider absorbs the change.

Likely to break, in rough order of probability:

  1. Database connections. Ten times the app instances means ten times the connection demand against a fixed limit. This is the most common first failure and the least visible in advance, because it appears as latency rather than as an error until the pool is exhausted.
  2. Database write throughput. Read replicas do not help. If writes are the bottleneck, the options are batching, queueing, or sharding — none of which are a month's work.
  3. A third-party API quota. A rate limit you have never approached becomes the constraint, and raising it needs a commercial conversation with lead time.
  4. A hot key or hot partition. Aggregate capacity is fine; one shard, one row or one cache key takes disproportionate traffic. Campaigns concentrate traffic on exactly one product, which is the worst case for this.
  5. Locks and contention. Contended row locks or a distributed lock degrade non-linearly: fine at 1,000, catastrophic at 10,000.
  6. Autoscaler lag. A campaign spike arrives in minutes; instance start plus warm-up takes several. The system fails during the ramp even though steady state would be fine.
  7. Anything single-instance — a scheduler, a cron worker, a stateful component someone forgot is not replicated.

How to find out

Model first, cheaply. Little's Law on each tier: at 10,000 requests per second and the measured per-tier latency, how many concurrent operations are needed, and how does that compare to the pool sizes and connection limits configured? This is arithmetic and it finds the connection-limit problem in an afternoon.

Then load test, properly. Production-like data volume (or the caches lie), a realistic request mix including the campaign's concentration on one product, and a ramp rather than a step so the autoscaler behaviour is exercised. Push past the target until something breaks — the purpose is to find the first bottleneck, not to get a pass mark.

Then fix and re-test. Removing the first bottleneck reveals the second. Expect at least three rounds.

What a strong answer adds

A protection plan for the case where the estimate is wrong: rate limiting so excess traffic is rejected cleanly instead of degrading everyone, a queue in front of the write path, a static fallback page, and a pre-agreed decision about what to shed first. Marketing's forecasts are not SLAs.