practice

Prompt Versioning

Treating prompts as versioned, reviewed, tested and deployable artifacts rather than as strings edited in place.

llmchange-managementgovernance

A prompt is production logic. It determines behaviour as directly as code does, and in most early LLM applications it is the least controlled artifact in the system — edited by whoever has console access, with no history, no review, no test and no rollback.

The consequences are predictable. Nobody can say what changed when quality regressed. A change that improved one use case silently broke another. An incident cannot be rolled back because the previous text is gone. And the person who tuned it has moved teams.

Treating it as an artifact means: stored in version control or a system with equivalent history, changed through review, evaluated against a regression set before deployment, versioned so a specific version is deployed and can be pinned, and logged with each request so the output can be attributed to the exact prompt that produced it.

The related discipline is coupling prompts to model versions. A prompt tuned for one model version does not necessarily behave the same on the next, so a provider-side model update is an uncontrolled change to your application. Pinning model versions where the provider allows it, and re-running the evaluation set before moving, is what turns that from a surprise into a release.

The organisational question this raises usefully: who owns prompt changes? They are often authored by product or domain specialists rather than engineers, which is fine, provided the change still goes through evaluation and review rather than around it.