Prompt & Configuration Registry  ·  View 04 of 21  ·  People and journeys

Journey — Shipping a Prompt Change

From a support ticket to five percent of live conversations, and the one phase where confidence drops.

Editable source SVG draw.io All views
Prompt engineer billing assistant Goal — Change what the assistant says about refunds, and know it is safe before a customer sees it Trigger — Support reports the assistant apologising for charges the customer was never billed Done when — New wording live for 5% of conversations, with the old one one click away 1 · Notice outside the tool 2 · Draft 3 · Review 4 · Release ◆ moment of truth 5 · Watch What they do Reads the ticket Finds the digest Edits the refund block Writes the rationale Opens for review Answers a comment Promotes to prod Sets 5% rollout Watches refusal rate What the platform does Resolves digest → version Flattens, validates Diffs against live Runs the eval gate Blocks on threshold Signs, stores, points Fans out the notice Reports convergence Streams exposures How it feels Confident Fine Anxious Where it hurts Which prompt said that? Eval takes 11 minutes Is it live everywhere yet? Signal lags the change What answers it Digest on every answer Gate verdict bound to digest Convergence metric, view 14 Rollback is one action Journey — Prompt Engineer Ships a Wording Change The trough is Release: nobody trusts a change they cannot see land. View 14 is the answer. v 1.0 · owner Platform Architecture · d 2026-09

The trough, and what answers it

  • Release is where the sentiment line falls, and it is not a UX problem: the engineer cannot tell whether the change is live on the whole fleet, half of it, or none of it.
  • The answer is structural — the convergence metric on view 14 and the digest on every answer from view 12. Without them this phase is a guess dressed as a deploy.
  • The second pain, an eleven-minute eval, is accepted rather than solved: a gate that is fast enough to feel instant is a gate measuring something trivial.

Assumptions

  • Evaluation of a digest completes in ≈ 11 minutes at p50 (stated assumption; the number drives whether the gate is synchronous in the promotion flow or a precondition on it).
  • Guardrail signals for prompt quality lag the change by minutes to hours, which is why Watch has no automatic verdict in the MVP (ADR-15).

What the platform owes this journey

  • A digest on the answer, so the ticket can be traced to a version before anything is edited.
  • A diff against what is live, not against the previous draft.
  • Rollback reachable in one action from the same screen that did the promotion.