AI-Era Architecture
General material on architecting systems that include models.
3 to work through
-
intermediate Multiple choice
A client wants an assistant that answers questions from 50,000 internal documents which change weekly. RAG or fine-tuning? What actually determines the quality?
2 min answer -
advanced
An LLM feature that worked last week now gives worse answers. Nothing was deployed. How do you find out what changed, and what should have been in place?
2 min answer -
advanced
You are asked to give an internal AI agent access to the customer database, the ticketing system and outbound email so it can resolve support tickets. What is your response?
2 min answer
15 terms in this topic
AI Gateway
A shared proxy in front of model providers that centralises routing, keys, quotas, caching, logging and safety policy.
conceptContext Window
The maximum number of tokens a model can attend to in one request, holding the system prompt, history, retrieved context, tools and the answer.
conceptEmbedding
A dense numeric vector representing a piece of content, positioned so that semantically similar content sits nearby.
patternGuardrail
A deterministic check applied to a model's input or output, enforcing rules that cannot be left to the model itself.
patternHuman in the Loop
Requiring human review or approval at a defined point in an automated flow, chosen by the reversibility and cost of the action.
practiceLLM Evaluation
A repeatable measurement of whether an AI system's outputs are good enough, on cases that reflect the actual task.
protocolModel Context Protocol
An open protocol that standardises how AI applications connect to external tools, data sources and prompts.
patternModel Router
Directing each request to a model chosen by the task's difficulty, cost and latency budget, rather than sending everything to the largest model available.
conceptPrompt Injection
An attack in which text from an untrusted source is interpreted by the model as instructions rather than as data.
toolPrompt Registry
A versioned store of production prompts with their model bindings, parameters and evaluation results, so a prompt change is a reviewable, traceable, …
practicePrompt Versioning
Treating prompts as versioned, reviewed, tested artefacts rather than as strings edited in place.
patternRetrieval-Augmented Generation
Retrieving relevant documents at query time and putting them in the model's context, so answers are grounded in your data rather than in training data.
patternSemantic Cache
Caching model responses keyed by the meaning of the request rather than by its exact text, so near-duplicate questions are served without a model call.
patternTool Calling
Giving a model a set of typed function definitions it can request to invoke, with the application executing the call and returning the result.
toolVector Database
A store optimised for approximate nearest-neighbour search over high-dimensional embeddings.
Neighbouring topics
LLM Application Architecture
The shape of a production system with a model in the request path.
No content yetRAG Architecture
Retrieval, grounding, citation and the permissions RAG can enforce.
No content yetVector Databases
Approximate nearest-neighbour search, filtering and re-indexing.
No content yetEmbeddings
Dense representations, model coupling and the migration they imply.
No content yetChunking & Retrieval
Structure-aware splitting, hybrid search and why chunking dominates quality.
No content yetReranking
Cross-encoders improving precision more than a bigger embedding model.
No content yetModel Selection
Capability, latency, cost and the evaluation that decides between them.
No content yetAI Gateways
Centralised routing, keys, quotas, caching, logging and safety policy.
No content yetPrompt & Version Management
Prompts as reviewed, versioned, evaluated production configuration.
No content yetAgent Architectures
Loops, planning, memory and the boundaries an agent must not cross.
No content yetTool Calling
Typed tool interfaces, narrow parameters and per-tool authorisation.
No content yetMulti-Agent Systems
Coordination, hand-off and whether more agents actually help.
No content yetLLM Evaluation
Held-out sets, rubric judging, CI gates and production sampling.
No content yetAI Observability
Logging prompts, versions, retrieved context and cost per request.
No content yetGuardrails
Deterministic checks on input and output that fail closed.
No content yetPrompt Injection Defence
Breaking the private-data, untrusted-input, outbound-channel combination.
No content yetAI Cost Management
Token accounting, routing, caching and the context-window budget.
No content yetHuman in the Loop
Gating by reversibility and blast radius, and avoiding approval fatigue.
No content yetML Platform
Feature stores, training pipelines, registries and deployment.
No content yet