LLM Evaluation
Held-out sets, rubric judging, CI gates and production sampling.
Nothing written here yet
This topic is part of the curriculum but has no questions, cards or glossary entries so far. The rest of AI-Era Architecture may still cover what you are looking for.
Neighbouring topics
AI-Era Architecture
General material on architecting systems that include models.
LLM Application Architecture
The shape of a production system with a model in the request path.
No content yetRAG Architecture
Retrieval, grounding, citation and the permissions RAG can enforce.
No content yetVector Databases
Approximate nearest-neighbour search, filtering and re-indexing.
No content yetEmbeddings
Dense representations, model coupling and the migration they imply.
No content yetChunking & Retrieval
Structure-aware splitting, hybrid search and why chunking dominates quality.
No content yetReranking
Cross-encoders improving precision more than a bigger embedding model.
No content yetModel Selection
Capability, latency, cost and the evaluation that decides between them.
No content yetAI Gateways
Centralised routing, keys, quotas, caching, logging and safety policy.
No content yetPrompt & Version Management
Prompts as reviewed, versioned, evaluated production configuration.
No content yetAgent Architectures
Loops, planning, memory and the boundaries an agent must not cross.
No content yetTool Calling
Typed tool interfaces, narrow parameters and per-tool authorisation.
No content yetMulti-Agent Systems
Coordination, hand-off and whether more agents actually help.
No content yetAI Observability
Logging prompts, versions, retrieved context and cost per request.
No content yetGuardrails
Deterministic checks on input and output that fail closed.
No content yetPrompt Injection Defence
Breaking the private-data, untrusted-input, outbound-channel combination.
No content yetAI Cost Management
Token accounting, routing, caching and the context-window budget.
No content yetHuman in the Loop
Gating by reversibility and blast radius, and avoiding approval fatigue.
No content yetML Platform
Feature stores, training pipelines, registries and deployment.
No content yet