Remote Build Cache Hit Rate
also called Action Cache Hit Rate, Build Reuse Rate
The share of build and test actions served from a shared cache rather than recomputed, which determines whether remote execution speeds the inner loop up or merely adds network latency to it.
A platform team buys remote execution, and the median build halves. Three months later engineers say builds are slower than before. Both are true: the median improved and the worst case got worse, because a cache miss now pays input upload, queueing and a network round trip on top of the compute it was always going to do.
The number that distinguishes the two outcomes is the hit rate, and it is usually not measured because the build system reports wall-clock time by default and reuse only if asked.
Why it matters
The inner loop runs dozens of times a day per engineer, so a change in its distribution multiplies. Take 200 engineers hitting a cold build twice a day at 7 minutes: about 47 engineer-hours of waiting every working day, on the order of six full-time engineers a year. Whether a remote cache removes that or merely moves it depends entirely on reuse.
The hit rate also diagnoses something a timing chart cannot. A low hit rate is almost never a cache problem; it is a hermeticity problem. An action whose key includes a timestamp, an absolute path, an unpinned toolchain or an environment variable can never match a previous run, so the cache is correct and useless. The metric therefore points at the build definition rather than the infrastructure.
Implementation patterns
- Report hit rate per action type, not as one fleet number. Compilation, test execution and linking behave differently and the remedy differs.
- Measure the first build after a branch switch separately. That is the cold case engineers complain about, and the only one the platform can really improve.
- Hunt non-determinism rather than tuning the cache. Usual culprits: timestamps embedded in artefacts, absolute paths in debug info, locale and clock, network access during a step, and filesystem-dependent test ordering.
- Populate the cache from CI, so every developer's first build hits the main-branch result rather than only what they built themselves.
- Keep a local escape hatch - a flag forcing a purely local build, tested in CI - so a cache outage is slow rather than total.
- Watch for poisoning. A key that omits a real input returns a stale artefact that builds cleanly and fails in production, which is worse than any slowness.
Industry example
At BazelCon 2024, a Canva engineer presented work on running web page builds and tests remotely on a remote-execution backend supporting action deduplication, reporting a reduction of roughly 25% in the compute needed to build and test those pages, with further gains expected. That shape is typical: the first win from remote infrastructure is often deduplication and reuse across many concurrent builds rather than raw parallelism, and it is only available because the actions are hermetic enough to be recognised as identical.
Failure scenarios
- Tail regression. p50 halves, p99 doubles, and the engineers who switch branches most - usually the senior reviewers - get the worst of it.
- Network dependency on the inner loop. The cache or executor is unreachable and nobody can build, turning a developer-experience investment into an availability incident.
- Quiet poisoning. An undeclared input means a cached output is reused when it should not be; the defect reaches production and is unreproducible locally.
- Cost inversion. A large rarely-reused repository uploads gigabytes of inputs per build and the remote bill exceeds the engineer time saved.
Trade-offs
| Choose remote caching and execution | Gains | Pays |
|---|---|---|
| With hit rate above roughly 70% | Cold builds become downloads; CI and laptop share work | A network dependency; worse p99 on misses; storage and egress cost |
| With hit rate below roughly 50% | Little | Overhead on nearly every action plus the operating cost of the cache |
The decision rule: remote execution pays when the ratio of parallelisable actions to local cores is large and the build is hermetic enough to reuse. Below that, buy faster laptops.
When not to use it
A repository that builds in 90 seconds on a laptop does not need a remote executor, and adding one buys a network dependency for the inner loop in exchange for nothing measurable. Skip it where the build cannot be made hermetic in the time available: an unpinned build with a remote cache is a correctness hazard, not a speed-up. It is also the wrong first investment when the dominant wait is elsewhere - if engineers wait 25 minutes for a test environment and 4 minutes for a build, the environment is the problem, and build infrastructure is an expensive way to avoid looking at it.
Interview question
Q: "You added a remote build cache. Median build time halved and complaints went up. Diagnose it, name the number you would put on a dashboard, and give the condition under which you would turn the cache off again."
What a strong answer covers: the bimodal distribution and why the median is blind to it; the miss path paying upload, queue and round trip; hit rate per action type and the post-branch-switch cold build as the dashboard numbers; non-hermetic actions as the root cause rather than cache sizing; and the turn-off condition - a hit rate that will not rise above about 50%, or a build short enough that network overhead dominates.
Quick check
Quiz: A remote cache halves median build time and engineers say builds got slower. What is happening? Misses now pay input upload and a network round trip on top of the compute, so p50 falls while p99 rises; the hit rate, not the timing chart, is the number that explains it.
Flashcard: What does a persistently low build cache hit rate usually indicate? — Not a cache problem but a hermeticity problem: actions whose keys include a timestamp, an absolute path or an unpinned toolchain can never match a previous run.