Adversarial Load Profile

Most load tests replay a representative period of production traffic at a multiple. That works when the future looks like the past at higher volume, and fails for the events that bring systems down, because a contested event changes the shape of the traffic and not only its size.

An adversarial load profile models the population rather than the aggregate: who is arriving, what they do when rejected, and how fast they come back. On a flash sale or a limited registration, clients compete with each other, many are automated, and failure makes them try harder. The profile reproduces that feedback loop somewhere it can be observed.

Why it matters

A proportional replay gets three things wrong at once, and each hides a different failure.

The derivative. Demand goes from background to peak within a second, so no warm-up, autoscaler reaction or pool growth curve is exercised. The mix. Traffic collapses onto one or two endpoints - access-code validation, inventory hold - while the browse paths that carried the replayed load sit idle. The amplification. A client that fails and retries offers more load than one that succeeds, so a system stable at 4x cooperative traffic can be unstable at 1.5x contested traffic.

Implementation patterns

• Drive by arrival rate, not by a fixed pool of virtual users. A closed pool waits for responses and reduces offered load exactly when the system slows - the most common way a load test reports a comfortable result for a system that would collapse.
• Parameterise populations separately: entitled clients that will succeed, unentitled clients that will be rejected, and automated clients looping with no backoff, each with its own arrival function and retry behaviour.
• Make retry amplification explicit - a queued human refreshing every 2 to 3 seconds, a bot an order of magnitude faster - and report the ratio of offered to useful requests.
• Put the admission mechanism under test, not the storefront behind it: the waiting room, the queue token, the rate limiter and the code validator decide the outcome.
• Run ordinary traffic concurrently and assert its latency budget and zero oversells - the first complaint often comes from an unrelated customer.

Industry example

Ticketmaster's own November 2022 statement on the Eras Tour on-sale reports about 3.5 billion total system requests, roughly four times its previous peak, attributing much of it to bot attacks and to fans without access codes. It records over 3.5 million pre-registrations with about 1.5 million codes sent, and estimates about 15% of site interactions experienced issues, including passcode validation errors that cost fans tickets already in their carts.

That account is the argument for the practice: the volume multiple was a number a capacity plan could carry, and the damage concentrated in one validation path under a population no replay of ordinary traffic contains.

Failure scenarios

• A closed virtual-user model, which under-reports offered load as latency rises.
• Think time copied from production, modelling calm users in an event with none.
• No model of rejected clients, usually the largest population and on the costliest endpoint.
• A test that stops at the storefront, leaving the admission mechanism unexercised.
• Synthetic holds consuming real inventory when the profile runs in production unshaded.

Trade-offs

This is expensive: a generator fleet large enough to produce the step, an inventory dataset of comparable size and distribution, traffic shading if any of it runs in production, and two engineers for several weeks, of which the first runs measure the harness. It also encodes assumptions about hostile behaviour that will be wrong in detail, so it buys a bounded answer and not a precise one.

What it buys is the only honest answer to "will the event hold", plus the amplification ratio that sets capacity and admission control. Against the revenue cost of a failed on-sale a few engineer-weeks is a small premium; against a routine seasonal peak it is not.

When not to use it

When the event is under roughly twice normal peak and the bottleneck is already known. A single-endpoint saturation test against that constraint answers the question in a day, and the elaborate profile is ceremony that delays the fix.

It is also wrong where clients are not contested - an internal batch window, a reporting peak, a scheduled sync - because there traffic really is cooperative and the replay model is accurate. The deciding question is whether failure makes clients retry harder.

Interview question

Q: "Your load test passes at four times last month's traffic and the on-sale still fell over at a similar multiple. What was the test structurally unable to see?"

What a strong answer covers: the step arrival versus a ramp; the mix collapsing onto the validation and hold endpoints; retry amplification from rejected and automated clients making failure an input to load; the closed model hiding the real offered rate; and the admission mechanism never being the system under test.

Quick check

Quiz: Why does a fixed pool of virtual users under-report a contested event? - Each user waits for a response before issuing the next request, so offered load falls as the system slows - the opposite of real client behaviour.

Flashcard: What number should an adversarial load test report to capacity planning? - The ratio of offered to useful requests at peak, which quantifies retry amplification.