Real user monitoring shows p75 largest contentful paint at 2.1 seconds - comfortably inside target - while a marketing analysis shows a real drop in conversion from organic mobile traffic. Which measurement artefact most likely explains the gap?

The first three things to look at

1. When does your RUM beacon fire? If it reports on load, or on a timer, every user who left before that point contributed nothing. The users you most need to measure are systematically excluded, and the metric improves as the experience gets worse.
2. Bounce rate by segment, against the same segments as the performance data. A rising bounce rate with a flat performance metric is the signature of this artefact.
3. Sample counts per segment, not just the percentile. Survivorship shows up as a falling sample count on the slow segment.

The diagnosis

This is survivorship bias, and it is self-concealing. As a page gets slower, more users abandon before the measurement is taken, so they drop out of the population, and the surviving population is faster on average. The metric moves in the opposite direction to the experience.

The fix is mechanical. Report on visibilitychange to hidden and on pagehide, which is what the modern web-vitals approach does, so a session that is abandoned still sends what it measured. Then the slow sessions are in the data, and the p75 rises to tell the truth.

Why the other options fail

• "p75 hides a badly affected segment." A genuine and common problem - a 5% segment failing badly is invisible at p75 - but it does not fit the evidence here, because the conversion drop is on a large identified segment. This is the answer that is right in a different incident.
• "Synthetic tests mixed into field data." This does corrupt field metrics, usually by making them look better, and it is worth ruling out. But synthetic traffic is filterable by user agent and the effect is a constant offset rather than something that tracks a conversion change.
• "Sampling too low." Low sampling widens confidence intervals; it does not bias the estimate. Noise and bias are different failures, and treating a biased estimate as a noisy one leads to collecting more of the same wrong data.

What a strong answer adds

That the same artefact affects the industry's shared datasets in a milder form: field data collected from browsers is limited to eligible, opted-in users and weights by page views, so heavy users count more than first-time visitors - exactly the opposite weighting from the one that matters for acquisition.

The alert that would have caught it earlier

Beacon count as a first-class metric, per segment, alongside the percentile. A percentile without its sample count is uninterpretable, and a falling sample count on a segment is the earliest available warning that the population being measured has changed.

When this is the wrong diagnosis

If your beacon already fires on pagehide - the default in the web-vitals library since 2020 - this artefact is largely closed and you should prefer the segment explanation instead. The decision rule is mechanical: check when the beacon fires before interpreting any field percentile. The cost of the fix is a slightly higher reported p75, which is a political cost rather than a technical one and is the real reason teams leave it unfixed.