Evidence-Producing Gate

In regulated delivery the blocker on automation is rarely the test. It is that the existing gate produces something an auditor accepts - a signed document, a named person - while the automated gate produces a green tick in a system whose history can be edited and whose retention is 30 days.

An evidence-producing gate is designed so that its record is the deliverable. The check runs and writes an entry naming the artefact digest, the claim verified, the verdict, what decided it and when, to a store the submitter cannot alter. Verification and the proof of verification are produced in the same action, which lets a human signatory be replaced rather than overruled.

Why it matters

Three things change once the record is the output. Automation becomes possible without losing the audit position, because what the auditor samples has got more complete - every release instead of a quarterly sample. Gate drift becomes visible: a gate that silently stopped running leaves a gap in the record, where a dashboard shows green. And approval becomes queryable: "which releases in Q3 shipped without the schema-compatibility claim".

This is not build provenance, which attests how an artefact was produced; this attests that a verification ran against it and what it decided. Confusing them produces a supply-chain programme that still cannot say whether anything was tested.

Implementation patterns

• Name the artefact by digest, never by tag. A record about release-2026-10 proves nothing if the tag moved.
• Enumerate claims explicitly, one record each: schema backward compatible, regulated screens render disclosures, payment totals correct on the five reference scenarios. A single "tests passed" record is not auditable because nobody can say what it covered.
• Write to an append-only store with regulatory retention, signed with a key the build job cannot read.
• Change the evidence format before changing the producer, so the switch to automation changes who writes the record and not what the auditor reads.
• Add a post-deploy digest assertion, because the gate certifies an artefact while the humans it replaced checked a running system.

Industry example

The clearest public reference point is the SLSA v1.0 specification, whose build track separates provenance that merely exists and may be unsigned (level 1) from provenance generated and signed by a hosted platform the user cannot influence (level 2). The principle it formalises - evidence is worth only as much as the submitter's inability to forge it - transfers directly to quality gates, and gives a staging plan: record first, then sign, then isolate the producer.

Change-control regimes demanding documented evidence of testing per release are satisfied far more cheaply by a machine-written record, provided it names the artefact and the claim.

Failure scenarios

• Evidence in the CI system's own run history, which is mutable and kept for weeks.
• A record naming a branch or tag, which cannot be tied to the artefact that shipped.
• One combined verdict covering an unknown set of checks, answering no specific question.
• The signing key available to the build script, making the signature decorative.
• An override flag that leaves no record, so the exception is invisible.

Trade-offs

The cost is infrastructure and discipline: a store with real retention, key management the build cannot subvert, and claims written precisely enough to be machine-checked. Precision is the expensive part - most teams discover their sign-off asserts two claims nobody had ever checked.

When not to use it

When nobody will ever ask. For an internal tool with no regulatory exposure and no cross-organisation trust boundary, the pipeline's own history is adequate and the machinery is ceremony. The trigger is an external reader: an auditor, a customer's security review, or a team that must trust your release without re-testing it.

It is also wrong where release frequency is very low. Two releases a year with a four-person-day manual pass costs eight person-days, which a quarter of evidence engineering never pays back. Spend on the record format alone and keep the humans.

Interview question

Q: "Your regulator accepts a signed spreadsheet from two testers per release. You want to automate the gate. How do you keep the audit position, and what is the point of no return?"

What a strong answer covers: enumerate the claims the sign-off actually asserts; change the evidence format while humans still produce it; run automated checks advisory and promote after a run of releases with no unexplained disagreement; the environment gap and the post-deploy digest assertion; and that the point of no return is retiring the last manual claim, because the human capability disappears within a quarter.

Quick check

Quiz: What distinguishes an evidence-producing gate from a passing build? - It writes a tamper-resistant record naming the artefact digest, the claim and the verdict, so the check can be demonstrated months later.

Flashcard: Why must a gate's evidence name the artefact by digest rather than by tag? - A tag can be moved after the check ran, breaking the link between what was verified and what shipped.