An engineer wants to change the system prompt of a live customer-facing assistant at 16:00 on a Friday. Tell me what has to already be true for that to be a routine change and what you would ask before approving it.

What the interviewer is testing

Whether you treat a prompt as a deployment artifact or as configuration. The Friday framing is the tell: a change is safe on a Friday when it is observable and reversible in minutes, and no amount of care substitutes for either.

The clarifying questions that change the answer

• Does this prompt drive a tool call or only text? A prompt that can trigger a refund, an email or a database write is a privileged change and never routine. A prompt that changes tone is.
• Is the model pinned to a dated snapshot or to a floating alias? If the alias floats, the prompt was tuned against a model the provider can replace without telling you, and behaviour can change with no deploy at all.
• What share of traffic does it touch, and can I ramp? A change that goes to 100% at once has no observation window.
• How is it rolled back? If rollback is a code deploy, the answer is no on Friday. If it is flipping a pointer in a registry, the answer can be yes.

What has to already be true

1. The prompt is a versioned artifact, reviewed like code, stored with the model id, temperature, tool schema version and output schema it was tested against. Those travel together or the version means nothing.
2. A regression suite runs in CI on every prompt change: 150 to 300 cases with checkable properties, not vibes. Does the output parse against the schema, does it cite a retrieved document, does it refuse the cases it must refuse, is the length within bounds. Assertions beat scores because they do not drift.
3. Staged rollout by traffic share, 5% for an hour, then 50%, then all. Compare the new version against the old on the same traffic rather than against yesterday.
4. Online guardrail metrics that need no labels: schema-parse failure rate, refusal rate, average output length, tool-call rate, escalation-to-human rate, retry rate. Each of these moves sharply on a bad prompt within minutes.
5. A kill switch that reverts by pointer, with the previous version kept warm.

Common weak answers

• "We test it manually on a few examples." Ten examples cannot detect a two-point regression, and the failures that matter are in the tail you did not think to try.
• "An LLM judge scores it." Useful as a signal and not sufficient as a gate. Judges drift with the judge model, favour longer answers, and prefer outputs from the same model family. Use them alongside assertions and calibrate them against human labels on a fixed set.
• "Prompts are not code." They are the most powerful untyped input in the system, and the one most often edited by whoever is closest to the complaint.

What a strong answer adds

The honest limit: the evaluation set covers what has already gone wrong. It cannot cover the long tail, so the release process must assume a miss and make detection and rollback fast rather than trying to make the gate perfect. A strong answer also names the organisational cost: a prompt registry means product managers stop editing strings in a file, and that is a change in how the team works, not only in tooling. Where this is over-engineered: a prototype with no external users needs the pinned model and the rollback, and can skip the rest until someone depends on it.