Your release gate for a retrieval assistant is a 600-case evaluation suite with expected answers, committed to the same repository whose contents the assistant's retrieval corpus indexes nightly. After the move the gate's pass rate went from 86% to 97% in one release with no model change and no prompt change. The team reads it as progress and raises the ship threshold to 95%. What has the organisation bought and what has it paid, and when does the bill arrive?

What was gained

Convenience that is worth something: the suite runs in the same job as the tests, failures are visible to the engineer who caused them, and nobody needs an access request to see a failing case. Co-locating evaluations with code is why gates get run at all.

What was paid

The gate stopped measuring the assistant and started measuring retrieval of its own answer key. The corpus now contains the expected answers, so the system can satisfy a case by finding it. An 11-point jump with no model change and no prompt change is not a capability improvement; it is the signature of a contaminated suite.

The cost compounds through the threshold. A 95% bar on a contaminated suite is easier than the 86% bar was on a clean one, so the gate is now weaker than the day before it was automated, while reading as stricter in the release record.

The detector, and the number to watch

Put twenty canary cases in the suite whose expected answers are unique strings that appear nowhere else and are deliberately not derivable from the corpus. If the system returns them, the key is reachable. Watch pass rate per case cohort rather than the headline: a jump concentrated in cases whose expected answers are long and distinctive is leakage, while a real improvement moves the difficult cohort too.

When the bill arrives

At the first incident the gate was green for. The suite's score is uninformative long before anyone notices, and the discovery usually comes from outside - complaint rate, a support macro that stops matching, an escalation about a wrong answer - weeks after the release that caused it.

The custody design

• The authoritative suite lives outside everything the system can read: a separate store, written by the evaluation owner, with a hash manifest recorded in the release record so the version that gated a release is identifiable a year later.
• A rotating slice. Replace roughly 20% of cases each quarter and keep a held-back slice that is never shown to anyone building the system, because a suite that engineers can iterate against becomes a training target even without a leak.
• Independent holders are the strongest form. Anthropic's published Claude system cards label which reported evaluations were run by third parties rather than in-house, and the UK and US AI Safety Institutes ran a joint pre-deployment evaluation of the upgraded Claude 3.5 Sonnet, published in November 2024, covering biological capabilities, cyber capabilities, software and AI development, and safeguard efficacy. The party being measured could not see those cases in advance, which is what made the result mean something.

The trade is explicit: custody costs you debugging speed. An engineer with a failing case now needs a redacted reproduction or an owner's help, which adds hours to a fix.

When this is the wrong answer

Contamination only matters when the system can read its own inputs. Unit tests of a deterministic function belong next to the code and always will. And proportionality applies to custody: if the suite informs a dashboard rather than blocking a release, the repository is the right home and the custody machinery is waste. Custody earns its cost in proportion to the authority the gate holds.