Beacon Survivorship

The field dashboard shows p75 largest contentful paint at 2.1 seconds, comfortably inside target. Conversion from organic mobile traffic is down 12% quarter on quarter. Both numbers are correct, and the performance number is being read wrongly.

If the measurement beacon fires on load, or on a timer, every user who left before that point contributed nothing. The population being measured is the population that waited, and the users you most need to see are systematically absent.

Why it matters

This is not noise. It is bias, and it points in the most damaging possible direction: as a page gets slower, more users abandon before measurement, so the surviving population is faster on average and the metric improves. A regression can make the dashboard look better.

That property makes it self-concealing. A noisy metric announces itself with wide confidence intervals; a biased one looks clean and confident and is wrong. Teams then invest against the wrong picture, and the conversion data and the performance data tell contradictory stories that nobody can reconcile.

The same effect appears in milder forms in shared field datasets: browser-collected corpora include only eligible, opted-in users and weight by page views, so heavy users count more than first-time visitors - exactly the wrong weighting for judging acquisition.

Implementation patterns

• Send on visibilitychange to hidden and on pagehide, not on load. This is what the modern web-vitals approach does, and it is the single change that closes most of the gap: an abandoned session still reports what it measured.
• Use a transport that survives unload. A keepalive fetch or the beacon API, because an ordinary request issued during teardown is frequently dropped.
• Report beacon count per segment as a first-class metric, beside the percentile. A percentile without its sample count is uninterpretable.
• Alert on a falling sample count for a segment. It is the earliest available warning that the measured population has changed.
• Cross-check against a population that cannot opt out of being counted - server-side request counts, bounce rate from analytics - and investigate when the two disagree.
• Record partial measurements. A session abandoned at 6 s with no LCP yet is itself a data point: "at least 6 s" is far more useful than silence.

Industry example

The shift in web performance tooling from load-time reporting to reporting on page hide happened precisely because of this effect: measuring at load cannot capture sessions that never reach it, and the browser performance community moved the recommended reporting point to visibility change around 2020 so that abandonment is represented. The general form is the oldest known statistical trap in engineering measurement, documented since wartime survivorship analyses in the 1940s: you are looking at the aircraft that came back.

Failure scenarios

• A performance improvement that shows as a regression, because the fix keeps slow users around long enough to be measured and they drag the percentile up. Teams have reverted good work over this.
• Two dashboards that disagree - performance green, conversion red - with no resolution because the underlying populations differ.
• A segment that quietly vanishes from the data as its experience degrades, so the alert never fires.
• Optimising the fast path. Investment follows the visible data, which describes users who are already fine.
• Sampling treated as the cause. Increasing the sample rate collects more of the same biased data and changes nothing.

Trade-offs

Fixing this makes your numbers worse. The reported p75 rises, sometimes substantially, and someone will have to explain a regression that is actually an improvement in honesty. That political cost is the real reason the fix is deferred, not the engineering, which is a few lines.

There is also a genuine technical cost: reporting at page hide gives less complete data per session, because some metrics are still settling, and the transport is less reliable than a normal request. The answer is to accept partial records rather than to wait for complete ones.

When not to use it

When the beacon already fires at page hide, this artefact is largely closed, and reaching for it as an explanation will send an investigation the wrong way - prefer the segment-dilution explanation, where a small badly affected cohort is hidden inside an aggregate percentile.

It is also not the right lens for authenticated long-session applications, where users do not abandon during load in meaningful numbers and the interesting measurements happen well after the page is up. The bias is strongest exactly where acquisition matters: unauthenticated first visits from search and social traffic.

Interview question

Q: Your field performance data has improved for three consecutive months and your bounce rate has risen in the same period. What are the possible explanations, and how would you distinguish between them?

What a strong answer covers: listing the candidates - genuine improvement with an unrelated bounce cause, a traffic-mix change, and survivorship - then distinguishing them with evidence rather than argument · checking beacon count per segment, which falls under survivorship and holds under a genuine improvement · checking when the beacon fires and what transport it uses · comparing against server-side request counts, which cannot be biased by abandonment · looking at the traffic-source mix, since a shift towards returning users improves performance and bounce simultaneously for benign reasons · and stating the expected outcome of the fix, that the reported percentile will get worse, so the team is not surprised into reverting it.

Quick check

Quiz: Why is increasing the RUM sample rate not a fix for this? Because sampling affects noise, not bias - a larger sample of a population that excludes abandoned sessions estimates the same wrong number with tighter confidence intervals.

Flashcard: Which single metric makes a field percentile interpretable? — The beacon count for the same segment. A falling count with an improving percentile is the signature of survivorship.