Portfolio ↗

Prompt & Version Management

Prompts as reviewed, versioned, evaluated production configuration.

4Questions
5Flashcards
2Terms
Drill this topic All of AI-Era Architecture
Questions

4 to work through

  1. intermediate

    A platform's prompts are embedded in application code and changed frequently. What problems arise, and how should prompts be managed?

    2 min answer
  2. intermediate

    An engineer wants to change the system prompt of a live customer-facing assistant at 16:00 on a Friday. Tell me what has to already be true for that to be a routine change and what you would ask before approving it.

    3 min answer
  3. intermediate

    Answer quality degraded overnight. No code was deployed. What are the candidate causes?

    2 min answer
  4. advanced

    140 prompts live as string literals across nine services. You must move them into a central prompt registry with no quality regression and no big-bang cutover, while every team keeps shipping. What is the sequence, and where can it go wrong?

    3 min answer
Terminology

2 terms in this topic

practice

Prompt Regression Suite

A set of test cases with expected properties, run against a prompt on every change, to detect quality regressions before deployment.

 practice

Prompt Versioning

Treating prompts as versioned, reviewed, tested and deployable artifacts rather than as strings edited in place.
AI-Era Architecture

Neighbouring topics

AI-Era Architecture

General material on architecting systems that include models.

7 quiz 19 cards 13 terms

LLM Application Architecture

The shape of a production system with a model in the request path.

4 quiz 10 cards 3 terms

RAG Architecture

Retrieval, grounding, citation and the permissions RAG can enforce.

6 quiz 9 cards 4 terms

Vector Databases

Approximate nearest-neighbour search, filtering and re-indexing.

4 quiz 6 cards 2 terms

Embeddings

Dense representations, model coupling and the migration they imply.

3 quiz 8 cards 3 terms

Chunking & Retrieval

Structure-aware splitting, hybrid search and why chunking dominates quality.

5 quiz 6 cards 3 terms

Reranking

Cross-encoders improving precision more than a bigger embedding model.

3 quiz 7 cards 3 terms

Model Selection

Capability, latency, cost and the evaluation that decides between them.

5 quiz 7 cards 3 terms

AI Gateways

Centralised routing, keys, quotas, caching, logging and safety policy.

4 quiz 6 cards 2 terms

Agent Architectures

Loops, planning, memory and the boundaries an agent must not cross.

4 quiz 6 cards 2 terms

Tool Calling

Typed tool interfaces, narrow parameters and per-tool authorisation.

4 quiz 8 cards 3 terms

Multi-Agent Systems

Coordination, hand-off and whether more agents actually help.

4 quiz 6 cards 2 terms

LLM Evaluation

Held-out sets, rubric judging, CI gates and production sampling.

4 quiz 9 cards 2 terms

AI Observability

Logging prompts, versions, retrieved context and cost per request.

4 quiz 6 cards 4 terms

Guardrails

Deterministic checks on input and output that fail closed.

4 quiz 8 cards 3 terms

Prompt Injection Defence

Breaking the private-data, untrusted-input, outbound-channel combination.

4 quiz 11 cards 5 terms

AI Cost Management

Token accounting, routing, caching and the context-window budget.

5 quiz 9 cards 3 terms

Human in the Loop

Gating by reversibility and blast radius, and avoiding approval fatigue.

4 quiz 5 cards 3 terms

ML Platform

Feature stores, training pipelines, registries and deployment.

4 quiz 12 cards 4 terms