Review this. Prompts live in a database table and are edited through an internal web interface by product managers. There is no version history. An A/B test assigns users randomly between a `current` row and an `experiment` row. Outputs are logged with the user id and the response text. What would you remove, what would you change, and what would you leave alone?

What is actually required

Two things, and the design provides neither: the ability to say which prompt produced a given output, and the ability to return to a prompt that worked. Everything else on the list is downstream of those.

What I would remove, and why it is safe to

• The A/B test, as built. Outputs are logged with the user id and the response text and not with the prompt version, and the prompt rows are edited in place. So an edit during the experiment silently changes one arm mid-flight, and nothing in the log distinguishes before from after. The experiment cannot be analysed, which means it is producing a number that will be believed. Removing it is strictly better than keeping it until provenance exists.
• Direct edits to the row production reads. Not the editing interface - the path from a keystroke to live traffic with no step in between.

The one change that matters

Immutable prompt versions, with the version identifier recorded on every output. Concretely: editing creates a new version with a content hash; a pointer (live, canary) names which version serves; every inference record carries that identifier alongside the model name, model version and sampling parameters.

This single change makes the rest possible: an experiment becomes analysable because each response is attributable to an arm; a regression becomes diagnosable because you can diff the two versions; and a rollback becomes moving a pointer rather than reconstructing text from memory. Without it, every quality investigation ends in "we think someone changed the prompt".

It also closes a failure that has nothing to do with prompts: a model provider updating a model behind an unversioned alias produces exactly the same symptom, and only a record of model version alongside prompt version tells the two apart.

What I would leave, even though it looks odd

• Product managers editing prompts. They are frequently the right authors - the prompt encodes product intent, not engineering - and routing every wording change through a code review is how teams end up with a six-week cycle on a one-line change. Keep the interface; put a promotion step after it.
• The database as the store. A prompt is data with a lifecycle, and files in a repository make staged rollout and non-engineer authoring harder. Immutability is the requirement; a version-control system is one way to get it, not the only one.

What I would add after the one change

A small regression set - 30 to 60 cases with expected properties - run automatically on a new version before the pointer can move to it. It is the cheapest control available, it catches the instruction-deleted-by-accident class immediately, and it gives the promotion step something to gate on other than someone's judgement.

What it costs, and the rule to apply

The honest cost is friction: a promotion step adds minutes to what is currently a keystroke, and that friction lands on the people who edit most. Pay it only where it buys something - the decision rule is that any prompt serving production traffic needs an immutable version and a recorded identifier, while a draft being iterated in a sandbox needs neither. Storage is not the constraint: a version identifier and a hash are tens of bytes per inference record, against the response text already being stored. Versioned configuration with an environment pointer has been standard practice in feature-flag and deployment tooling since the mid-2010s, and model-risk guidance published since 2023 asks what produced a given decision - which requires exactly this record.

How I would argue this in the review

With a question rather than a critique: pick yesterday's worst response and ask which prompt produced it. Nobody will be able to answer, and that demonstration does more than an argument about process. Then offer the trade - keep the self-service editing, add provenance and a promotion gate - so the proposal gives product managers something rather than taking it away.

Common weak answers

• "Move the prompts into the repository and require a pull request." Gets immutability and loses the authors; the result is a six-week cycle on a one-line wording change, which teams then bypass.
• "Log the full prompt text with every response." Storage balloons, comparisons become string diffs, and outputs still cannot be grouped by version.
• "Let the experiment finish, then add versioning." The experiment's result is already unattributable, and shipping on it is the actual risk.
• "Add an audit table with a trigger." Records that an edit happened without linking any output to any version, which is the half that matters.