advanced 4 min answer

Review this supply-chain programme. Every build emits an SBOM and a signed provenance attestation. Admission policy verifies the attestation before a pod starts. Nothing records which artefact digest is running in which environment at which time. A critical vulnerability is announced in a transitive dependency. What would you remove, what would you change, and what would you leave alone?

sbomprovenanceattestationinventoryvulnerability
Show the full answer Hide the answer

What is actually required

During a vulnerability response, three questions get asked in this order: is it in anything we build, is it running anywhere right now, and how do we get it out. This programme answers the first well, the second not at all, and the third only as fast as the second can be answered.

That ordering decides the critique. An SBOM per build is a statement about an artefact. The question being asked is about a deployment, and no quantity of per-build metadata answers it without a record linking artefacts to places.

What I would remove

SBOM generation and retention for every build. A large repository produces thousands of builds a month, almost all of which are never deployed anywhere. Storing and indexing their SBOMs costs real money and makes the index slower to query, which is a direct cost during the incident when the query matters.

Generate and retain SBOMs for artefacts that are promoted — anything that reaches a shared environment — and keep the per-build generation as an ephemeral artefact attached to the build for a short window. The signal is unchanged and the index shrinks by an order of magnitude.

I would also remove any policy rule that verifies an attestation exists without verifying who signed it. An attestation checked for presence rather than for a trusted builder identity is ceremony: anything that can produce an attestation passes.

The one change that matters

A deployed-artefact ledger. A record, written by the deployment mechanism, of which artefact digest was deployed to which environment and cluster, when, by what, and when it stopped running. Append-only, queryable, with a retention long enough to cover the longest-lived running workload.

With it, the vulnerability response is one query against the ledger joined to the SBOM index, and the answer includes the environments and the time ranges. Without it, the answer is assembled by asking teams, which is the step that turns a 20-minute response into a two-day one.

The ledger also answers the audit question that provenance alone cannot: "prove the container running in production is the one that passed scanning" needs the running digest, not the built one.

What I would leave alone, even though it looks like overhead

Admission-time verification. It is easy to argue it is redundant once a ledger exists, and it is not: the ledger is a record and admission is a control. The record tells you what happened; the control stops the wrong thing happening. Keep it, and strengthen it to check the signing identity.

The attestation format itself, even though almost nothing has ever been rejected. Its value is not the rejections; it is that the build's inputs and builder are recorded in a way a third party can check. The xz-utils backdoor disclosed in March 2024 (CVE-2024-3094) is the standing argument: the malicious macro was present in the released tarballs and not in the equivalent state of the public repository, so a pipeline that trusts a release archive and not a verified build from source has no way to see the difference.

What the team is probably missing beyond the ledger

Revocation. Knowing which deployments are affected is necessary and not sufficient; the response also needs a way to get a fixed artefact to every affected place quickly, and that is a rollout capability with its own measured throughput. "How long to get a patched image to 40 clusters" is a number that should be known before the day it is needed, and it is usually discovered then.

When not to build any of this

For a single application deployed to one environment by one team, the ledger is the deployment history your tooling already keeps, and an SBOM index is a query over one artefact. The machinery above is justified by scale and by multi-tenancy of responsibility: many teams, many clusters, and nobody with a global view. Below roughly 5 teams and 3 environments, prefer reading the deployment history to building a ledger.

Attestation verification is the exception worth keeping even when small, because it costs almost nothing once the build system emits it and it is the only control that distinguishes a verified build from a release archive someone uploaded.

How I would argue this in the review

With one question and a stopwatch: "a critical vulnerability is announced in a transitive dependency right now — show me the list of affected running workloads." The time taken is the finding, and it will be measured in hours. Then propose the ledger as the only change needed to make that minutes, and fund the SBOM retention reduction as the thing that pays for it.