An executive asks why the new system "feels slower" when your dashboards show average response time improved. How do you explain it?
Show the full answer Hide the answer
The average is the problem, and that is the explanation
Averages hide the tail. A system where most requests got faster and the slowest ten percent got much slower has a better average and a worse experience — because users remember the slow ones, and a single slow request in a session colours the whole session.
Show the percentile distribution rather than the average, and show it before and after. The picture usually makes the argument without further explanation.
Check what the dashboard is actually measuring
Two common gaps between the number and the experience:
Where it is measured. Server-side response time excludes network, rendering and client execution. The user's experience includes all of it, and a change that moved work to the client improves the server metric while making things worse.
Which requests. An average over all endpoints is dominated by the cheap frequent ones. The checkout submission that got slower is a small fraction of requests and all of the perception.
Then answer the real question
The executive is not asking about percentiles. They are asking whether the change was worth it. Answer that:
- What got better, with numbers.
- What got worse, with numbers.
- Whether the trade was deliberate or a surprise.
- What it would cost to fix the regression, and when.
What not to do
Do not defend the metric. "The dashboard says it is faster" is technically true and tells the executive that you are measuring something other than what they care about — which is a more damaging conclusion than the latency regression itself.