A platform team reports that the median build is 48 seconds and calls the inner loop healthy. Engineers keep complaining. Per-developer instrumentation shows two modes - about 70% of runs finish under a minute and about 30% take 6 to 9 minutes after a branch switch or a dependency bump. Why does the median hide the problem, and what should the platform measure instead?

Why the median is the wrong statistic here

A median describes the middle run. When a distribution has two modes, the median sits inside the common one and never moves, however bad the other mode gets. Slow runs could double in length and the reported 48 seconds would not change by a second.

The arithmetic says which mode matters. Take 200 engineers hitting the slow mode twice a day at 7 minutes each: that is 2,800 minutes, about 47 engineer-hours of waiting every working day, roughly six full-time engineers' worth of a year spent watching a progress bar. The fast mode, at 48 seconds a run, is noise next to it.

The mechanism behind the second mode

The slow mode is almost always a cache miss, and the two triggers in the data name the cause.

• A branch switch changes the inputs of nearly every action. Build systems key cached outputs on a hash of the inputs: source, compiler version, flags, and the transitive dependency graph. Switching branches changes files near the root of that graph, so everything downstream is recomputed even though most of the output would have been identical.
• A dependency bump invalidates by construction. Change one line in a lockfile and the hash of every action that depends on it changes. Nothing is reusable because nothing has the same key.

So the fast mode measures the warm case and the slow mode measures the cold case, and the platform only controls the cold case: a remote cache populated by CI, so that the first build after a branch switch downloads outputs rather than producing them.

What to measure instead

• Cache hit rate per action, broken out for the first build after a checkout. This is the number that predicts the slow mode, and it is the one the platform can move.
• The 90^th percentile of each engineer's slowest build each day, not the fleet-wide median. It tracks the experience people actually complain about.
• Count of builds over five minutes per engineer per day. A count is easier to set a target against than a percentile: "no engineer waits more than five minutes more than once a day."
• Time from git checkout to first green test, which is the real unit of work and includes the index or dependency-resolution step that build timing usually excludes.

When this is the wrong answer

If the distribution is actually unimodal, the median is fine and chasing percentiles is noise. Below roughly 20 engineers the instrumentation costs more than asking them, and the honest first step is to sit with four people for an afternoon. Prefer a remote cache only once the build is hermetic: unless every input is declared, actions capture the system clock, an environment variable or an unpinned toolchain, a cached output is reused when it should not be and the build fails in production rather than on the laptop. That failure mode is worse than slowness, and the correct first investment is input declaration.

Common weak answers

• "Buy faster laptops." The slow mode is recomputation of work that already exists elsewhere, not a shortage of cores. Faster hardware shortens 7 minutes to 5 and leaves the cause untouched.
• "Report the mean instead." The mean moves, but it describes nobody's experience and gives no target. A count of slow builds is more actionable than either average.
• "Tell engineers to switch branches less." The platform exists to absorb that constraint, not to push it back onto its users.