A test must sustain 2000 requests per second against a service whose p50 is 120 ms, with 1 second of think time per virtual user between requests. Roughly how many virtual users does the generator need and what breaks first?

The assumptions, stated

• Each virtual user runs a loop: one request, then think time, then the next.
• Cycle time per user is response time plus think time: 120 ms + 1000 ms = 1.12 s.
• The generator is closed-loop, so a slow service reduces offered load rather than queueing more.

The arithmetic

requests per user per second = 1 / 1.12 s        ≈ 0.89
virtual users needed         = 2000 / 0.89       ≈ 2240
in-flight requests at the service = 2000 × 0.12 s ≈ 240   (Little's Law)

Two different numbers, and conflating them is the usual mistake. 240 is the concurrency the service experiences. 2240 is the population the generator must simulate, because most of each user's cycle is spent thinking.

Why the other options fail

• 240 virtual users. That is in-flight concurrency, not users. Drive 240 users with a 1 s think time and you offer about 214 rps, an order of magnitude short, and the test will report the service as healthy at a load it never saw.
• 2000 virtual users because think time cancels out. Think time is what makes a closed-loop test resemble reality; removing it turns the test into a saturation test where each user's next request is gated by the previous response, so the arrival process is wrong in a way that hides queueing.
• 120 virtual users and the network. 2000 requests per second of a few KB is a handful of Mbit per second. Bandwidth is rarely the constraint in an API load test, and quoting it usually means nobody did the arithmetic.

What the number rules out, and what breaks first

The generator. 2240 virtual users on a thread-per-user runtime with 1 MB stacks is 2.2 GB of stacks before any request buffers, so an event-loop generator is required rather than optional. With short-lived connections, each source address has roughly 28000 usable ephemeral ports and sockets sit in TIME_WAIT for around 60 s, so a design that opens a new connection per request runs out of ports long before it runs out of CPU: reuse connections, or add source addresses.

Measure the generator before trusting the result. If generator CPU is above roughly 70% or its own latency histogram is skewed, the reported service latency includes generator queueing and is inflated. The honest test reports both.

When this is the wrong model entirely

For a service fronted by an open-loop arrival process — events, webhooks, IoT ingest — the closed-loop virtual-user model understates the harm of slowdown, because real senders do not wait politely. Use an open-loop generator with a fixed arrival rate there, and expect it to reveal coordinated omission that a closed-loop test cannot.