Build Once, Promote the Artefact

The rule that makes deployment pipelines trustworthy is short: build the artefact once, then promote that exact artefact through every environment. Rebuilding for staging and again for production means the thing that was tested is not the thing that shipped, and every difference between the builds, a dependency that resolved differently, a base image that moved, a compiler flag, is a difference the tests never saw.

Separating the artefact from its configuration

If the artefact is identical everywhere, everything that differs between environments has to live outside it: endpoints, credentials, resource limits, feature flags, sampling rates. That separation is what makes the promotion meaningful, and it forces a useful discipline, since anything that cannot be expressed as external configuration is a genuine behavioural difference between environments that ought to be visible.

The artefact is addressed by digest, not by tag. A pipeline that promotes myservice:staging to myservice:prod by retagging has promoted a name; a pipeline that promotes a digest has promoted the bytes.

Where ML deployments break the pattern

The model is often built separately from the code that serves it. A serving image is built by CI; the model is produced by a training pipeline on a different schedule. Two artefacts must be promoted together, and if only one is versioned in the deployment, the pair that was tested is not the pair that runs. The clean resolution is a deployment manifest naming both by digest, promoted as one unit.

Model artefacts are too large to embed conveniently. A container image with a hundred-gigabyte model in it is slow to pull and slow to start. Referencing weights by digest from object storage keeps the image small at the cost of a second artefact whose immutability must be guaranteed separately.

Configuration expands to include behaviour. For an LLM system the prompt, tool definitions and generation parameters are configuration by mechanism and behaviour by effect. Treating them as environment configuration means production behaves differently from staging with no artefact change, which is exactly what the pattern exists to prevent. They belong in the versioned artefact even though they look like configuration.

Warm-up and validation must run against the promoted artefact. A smoke test that loads the model and runs a fixed set of inputs, comparing against recorded outputs, catches a corrupted or mismatched artefact before traffic reaches it. This is cheap and is the check most often omitted, because the artefact "already passed CI".

When it breaks

Environment parity is never complete. Production has traffic patterns, data volumes and dependency versions that staging does not. Build-once removes the artefact as a variable, which makes the remaining differences easier to reason about rather than eliminating them.

Secrets injected at runtime can change behaviour. A configuration value that switches a code path is not configuration, it is a build variant wearing configuration's clothes. Auditing which config values alter behaviour rather than merely parameterising it is worth doing once.

Promotion without re-validation assumes the environment is stable. An artefact validated in staging on Monday and promoted on Friday runs against a production whose dependencies may have moved. Re-running the smoke test at promotion time, in the target environment, is what covers that gap.

Emergency patches break the chain. A hotfix applied directly to production creates an artefact that never existed in the pipeline, and the next normal deployment silently reverts it. Any emergency path has to end with the fix flowing back through the pipeline, and treating that as optional is how the same incident recurs.