Build Provenance

A signature proves an artefact came from someone holding a key. Provenance proves how it was produced: the source commit, the build definition, the builder identity, the dependencies and their hashes, all attested and signed.

The distinction matters because the threat is not usually a forged signature. It is a legitimately signed artefact containing code that is in no repository, inserted by a compromised build machine or a compromised dependency.

Why it matters most for distributed software

A SaaS compromise affects the SaaS. A compromise of software distributed to thousands of organisations affects all of them, on machines holding source code and production credentials. That makes the build and distribution pipeline a higher-value target than production infrastructure — and it is frequently protected less well, because it is treated as internal tooling.

Implementation patterns

• Reproducible builds, so that building the same source produces a bit-identical artefact and an independent party can verify correspondence. This is the only defence against a builder inserting code that appears nowhere in the source.
• Signing as a separate stage from building, with the key in hardware and never available to build code. If the build can reach the signing key, compromising the build compromises signing.
• Attestation covering the dependency set, with pinned versions and verified hashes fetched from an internal mirror rather than from the public internet at build time.
• Pipeline definitions from the trusted branch, never from the contribution being built — otherwise a contributor who edits the workflow bypasses every other control.
• Consumer-side verification, since provenance that nobody checks is documentation.
• An inventory of what is in each release, so "are we affected by this dependency vulnerability, and in which versions" takes minutes rather than days.

Industry example

Developer tooling vendors, package registries and container base image publishers all sit in this position: their artefacts execute inside customer environments with high privilege, and the update channel is a remote code execution channel by design.

That last point drives a specific set of controls beyond provenance: client-side signature verification on update, rollback protection so an attacker cannot force installation of an older vulnerable version, and staged rollout so a bad or malicious release reaches a bounded population before the fleet.

Alongside the technical controls, the organisational ones matter more than teams expect: two-person review for changes to build and release infrastructure — held to a higher standard than application code — separation of duties between committing and releasing, and a tested emergency revocation and update path, because the response to a compromised release is a race and the mechanism must exist beforehand.

Failure scenarios

• Signing keys accessible to build jobs, collapsing two controls into one.
• Provenance generated but never verified by consumers or by the release process itself.
• Dependencies fetched at build time from public registries, exposing the build to dependency confusion and to upstream compromise.
• No release inventory, so vulnerability response begins with an investigation.
• Update mechanisms without rollback protection, allowing forced downgrade to a vulnerable version.

Trade-offs

Reproducible builds are genuinely difficult to achieve — timestamps, path embedding, non-deterministic compilation, parallelism — and reaching bit-identical output can take substantial engineering. Full attestation adds pipeline complexity and a verification step that must not become a deployment bottleneck.

For software that runs only in your own infrastructure, the investment is harder to justify. For anything distributed to others, the asymmetry of consequence makes it one of the highest-return security investments available.

Interview question

"Your build system is compromised and a modified binary is signed and shipped. Which of your controls would have prevented it, which would have detected it, and how would you determine which customers received the bad artefact?"