Percentile Latency
Latency expressed as the value below which a given proportion of requests fall, used because averages conceal the behaviour that users notice.
An average of 100 ms is compatible with 95% of requests at 50 ms and 5% at one second. The 5% is what generates complaints, and the average is structurally incapable of showing it.
Read the percentiles as questions about population, not about severity: p50 is the typical experience; p95 and p99 are the slow tail; p999 is where the pathological cases live — cold caches, lock contention, garbage collection pauses, retries.
Two things people get wrong.
Percentiles cannot be averaged across instances or time buckets. The mean of three instances' p99 is not the p99 of their combined traffic, and it can be wrong in either direction. Aggregate the underlying histogram buckets, then compute the quantile — which is why bucket boundaries must be chosen deliberately and kept consistent.
A user does not experience one request. A page making 20 calls, each with a 1% chance of being slow, has roughly an 18% chance of containing at least one slow call. The user's experience is closer to your p99 than to your p50, which is why fan-out changes which percentile you must optimise.