Concept library
1015 concepts across 20 domains and 101 tracks. Each track is a coherent sequence — read it top to bottom or dip in wherever the gap is.
All domains
01Foundations
02Transformer Internals
03Training & Fine-Tuning
04Reinforcement Learning
05Inference, Systems & Hardware
06Applied LLM Engineering
07Reasoning, Evaluation & Safety
08Multimodal & Applications
09Classical ML & Statistical Learning
10Causal Inference & Experimentation
11Time Series & Forecasting
12Graphs, Recommenders & Structured Data
13Generative Modelling Beyond Transformers
14Efficiency, Compression & Edge AI
15Search & Information Retrieval
16Data & Feature Engineering
17MLOps & Platform Engineering
18Security, Privacy & Adversarial ML
19Governance, Risk & Responsible AI
20Human-AI Interaction, Product & Economics
06
Applied LLM Engineering
Building things people use: retrieval, prompts, agents and production architecture.
6tracks
63concepts
652cards
8.2hreading
Retrieval & RAG Vector stores, hybrid retrieval and reranking, and when to retrieve instead of fine-tune. 14 concepts · 161 cards
- 01 Choosing and Adapting an Embedding Model Why the top of the MTEB leaderboard is a bad way to pick an embedding model, what dimension, context length, and asymmetry actually cost you in production, and when fine-tuning on your own hard negatives beats buying a bigger model.
- 02 Chunking Strategies for Retrieval Why the unit you index decides the ceiling on retrieval quality, how fixed, recursive, semantic and contextual chunking differ, and what each one loses.
- 03 Contextual Retrieval and Chunk Augmentation Chunks lose the context that made them meaningful; prepending a short LLM-written situating sentence to each chunk before embedding cuts retrieval failures by roughly a third, and the technique only became affordable because of prompt caching.
- 04 Fine-tuning vs RAG When to teach the model new behaviour vs when to retrieve fresh context at runtime.
- 05 Hybrid Retrieval - BM25 + Vector + Reranking Why pure vector search misses exact-match queries, how RRF combines lexical and semantic results, and where a cross-encoder reranker buys back the precision you lost.
- 06 Query Transformation for Retrieval Why the user's question is often a bad search query, and how rewriting, decomposition, multi-query fan-out and HyDE close the gap between how people ask and how documents are written.
- 07 RAG Evaluation and Groundedness How to separate retrieval failures from generation failures, which metrics actually diagnose each stage, and why groundedness is measurable while helpfulness mostly is not.
- 08 Reranking and Cross-Encoders Why a second-stage model that reads the query and document together fixes most retrieval failures, what it costs in latency, and how to size the candidate set.
- 09 Retrieval Augmented Generation The end-to-end RAG pipeline from chunking through retrieval, reranking, and grounded generation.
- 10 Vector Databases Compared - pgvector, Qdrant, Milvus, Weaviate, LanceDB A practitioner's guide to picking a vector store, weighing index trade-offs against the operational cost of running yet another database alongside your primary store.
- 11 ANN Indexes: HNSW, IVF and PQ How approximate nearest neighbour indexes trade recall for latency and memory, what HNSW, IVF-PQ, ScaNN and DiskANN each optimise for, and why recall is a knob rather than a property.
- 12 Agentic and Iterative Retrieval Why a single retrieve-then-read pass cannot answer questions whose second search depends on the first result, and how retrieve-reason interleaving, self-reflection tokens, and search agents trade latency and cost for multi-hop accuracy.
- 13 GraphRAG and Community Summarisation Why top-k retrieval cannot answer "what are the main themes in this corpus", how building an entity graph and pre-summarising its communities turns a global question into a map-reduce over summaries, and what that indexing bill buys you.
- 14 Late Interaction and Multi-Vector Retrieval How ColBERT-style models keep one vector per token instead of one per document, why MaxSim recovers most cross-encoder quality at index-time cost, and what the storage bill looks like.
Prompt Engineering In-context learning, chain of thought, structured output, compression and injection-aware design. 10 concepts · 94 cards
- 01 Chain of Thought Prompting Why telling the model to think step by step radically improves reasoning, and when it actively hurts.
- 02 Few-Shot and In-Context Learning Learning a task from a handful of worked examples placed in the prompt, with no weight updates, and the surprising evidence about what those examples actually teach.
- 03 Injection-Aware Prompt Design How to structure prompts that consume untrusted input so injection is harder, and why prompt design alone can never make an LLM injection-proof.
- 04 Multimodal Prompting What changes when part of the prompt is an image: token cost scales with resolution, ordering of image and text changes the answer, and the reliable failure mode is confident description of objects that are not there.
- 05 Prompt Chaining and Task Decomposition Splitting a hard task into a pipeline of simpler, individually-checkable prompts so each step can be validated, routed, and debugged on its own.
- 06 Structured Output Coercion How to coax reliable JSON, XML, and tabular output from a model using prompting alone, and why that gives you no hard guarantee the way constrained decoding does.
- 07 System Prompt Design What the system prompt actually buys you as a privileged, cache-stable, always-present segment, how to structure it, and why length in that slot costs more than it looks.
- 08 Automatic Prompt Optimisation Treating the prompt as a parameter to be searched rather than a string to be tweaked, using a metric, a dataset, and an LLM that proposes and critiques its own instructions.
- 09 Prompt Compression Cutting prompt tokens while holding task performance, via perplexity-based token dropping (LLMLingua) or learned gist tokens, and when prompt caching beats both.
- 10 Prompt Format Sensitivity Semantically identical prompts that differ only in separators, spacing, or option order can move accuracy by tens of points, which makes any single-format benchmark number a sample rather than a measurement.
Agents & Tool Use Function calling, ReAct loops, MCP, agent memory architectures and evaluation harnesses. 15 concepts · 169 cards
- 01 Agent Frameworks Compared LangGraph, CrewAI, AutoGen, and the OpenAI Agents SDK solve different problems; the harder question is whether you need a framework at all.
- 02 Agent Skills and Progressive Disclosure Packaging agent expertise as folders of instructions and scripts that load in layers, so a hundred specialisations cost a few hundred tokens until one of them is actually needed.
- 03 Agent-to-Agent Interoperability MCP standardises how one agent reaches tools and data; A2A standardises how two independently built agents discover each other and collaborate as peers, which is a different problem with a different failure surface.
- 04 Model Context Protocol (MCP) The open standard that replaces bespoke per-tool integrations with one protocol, so any compliant client can talk to any compliant server.
- 05 Planning and Task Decomposition in Agents Why an LLM that reasons well step by step still fails to produce a valid multi-step plan, and how decomposition, external planners, and replanning close the gap.
- 06 Tool Use and Function Calling How models invoke external tools to fetch data, run code, and take actions in the world.
- 07 Agent Evaluation Harnesses Single-output accuracy says nothing about an agent that takes thirty steps; evaluating agents means scoring trajectories, environment state, and reliability across runs.
- 08 Agent Memory Architectures An agent whose only memory is its context window is amnesiac between sessions; persistent memory is the architecture that decides what to keep, where, and how to retrieve it.
- 09 Agentic AI and ReAct From single tool calls to multi-step agents that plan, act, observe, and recover from errors.
- 10 Code Execution as a Tool Interface Instead of calling tools one at a time through the context window, the agent writes code against tool APIs in a sandbox, which cuts both tool-definition overhead and intermediate results out of the token budget.
- 11 Computer-Use Agents: Operating a GUI Through Pixels How agents that click and type on a real desktop differ from tool-calling agents, why GUI grounding is the bottleneck, and what OSWorld measured that API benchmarks cannot.
- 12 Durable Agent Execution and Recovery Long-running agents fail mid-task for mundane reasons, so the loop needs checkpointed state, idempotent side effects, and the ability to resume from a step rather than restart from the prompt.
- 13 Long-Horizon Agent Reliability Why per-step accuracy compounds into task failure, how METR's time-horizon metric reframes agent capability, and which architectural moves actually raise the exponent.
- 14 Orchestrator-Worker Subagent Architectures A lead agent decomposes a task and spawns subagents with clean context windows that explore in parallel and return compressed summaries, buying breadth and context isolation at a large token cost and a coordination risk.
- 15 Sandboxing and Least Privilege for Agents Why agent security has to be enforced outside the model, how capability scoping and human-in-the-loop gates work, and what the CaMeL design proves about the limits of prompting.
Claude Certified Architect Agentic loops, coordinator-subagent designs, tool interfaces and reliability patterns for Claude. 10 concepts · 82 cards
- 01 Agentic Loops and stop_reason Handling The agentic loop lifecycle - sending requests, inspecting stop_reason, executing tools, and appending results. The foundation of every autonomous Claude agent.
- 02 CLAUDE.md Configuration and Claude Code Workflows CLAUDE.md hierarchy, .claude/rules/ with glob patterns, custom commands and skills, plan mode vs direct execution, and CI/CD integration.
- 03 Cost and Throughput Engineering on the Claude API The levers that actually move an LLM bill — effort, batch processing, model tiering, task budgets, and deferred tool loading — what each one costs in latency or quality, and why measuring before tuning is not optional.
- 04 Programmatic Enforcement vs Prompt-Based Guidance When to use hooks and programmatic prerequisites for guaranteed compliance versus system prompt instructions for probabilistic guidance.
- 05 Prompt Engineering and Structured Output Patterns Explicit criteria, few-shot prompting, tool_use with JSON schemas, validation-retry loops, and the Message Batches API.
- 06 Tool Interface Design and MCP Integration Writing effective tool descriptions, structured error responses, MCP server scoping, and the distinction between MCP tools and resources.
- 07 Context Lifecycle: Editing, Compaction, and Memory Three different mechanisms manage a long-running Claude agent's context — clearing stale tool results, summarising history server-side, and persisting files across sessions — and choosing wrongly between them is one of the most common architecture mistakes.
- 08 Context Management and Reliability Patterns Context preservation across long interactions, escalation decision-making, error propagation in multi-agent systems, and information provenance.
- 09 Multi-Agent Coordinator-Subagent Architecture Hub-and-spoke multi-agent design with coordinator delegation, isolated subagent context, parallel execution, and iterative refinement loops.
- 10 Prompt Caching Economics for Claude Agents Caching is a prefix match, so one interpolated timestamp can make an entire agent uncacheable; the architecture decisions that matter are ordering and stability, not where you place the breakpoint markers.
LLM Application Architecture Routing, fallbacks, caching layers, structured state, and the reference shapes production LLM apps take. 7 concepts · 74 cards
- 01 Batch and Asynchronous LLM Workloads How provider batch APIs trade a day of latency for half the price, how to shape traffic under token-bucket rate limits when you cannot wait, and why every asynchronous LLM job needs idempotent writes and a reconciliation step.
- 02 Document Ingestion and Parsing Pipelines Why turning PDFs into text is the silent ceiling on retrieval quality, how layout-model pipelines and vision-language parsers differ in cost and failure, and how to re-ingest a changing corpus without reprocessing all of it.
- 03 Fallbacks, Timeouts and Degradation How to build an LLM feature that survives a provider outage, why the usual retry patterns need adjusting for generation, and what degrading gracefully looks like when the core capability is unavailable.
- 04 Model Routing and Cascades Why sending every request to the strongest model is usually wrong, the two routing patterns and what each requires, and the economics that decide whether routing pays.
- 05 Caching Layers for LLM Applications The four distinct caches in a mature LLM system, what each requires to be correct, and why semantic caching is the one that is dangerous.
- 06 Structured Output and Constrained Generation Why parsing free text is the wrong integration point, how constrained decoding guarantees valid syntax, and what a guarantee about form does not give you about content.
- 07 Where State Lives in an LLM Application Why the context window is the worst place to keep state, the four stores a mature application actually uses, and the assembly step that decides what the model sees.
AI for Software Engineering Code models, repository context, patch generation, test-driven agents and SWE benchmarks. 7 concepts · 72 cards
- 01 Automated Program Repair and Fault Localisation How spectrum-based fault localisation ranks suspicious code from test coverage, why generate-and-validate repair overfits weak test suites, and how LLM pipelines such as Agentless rebuilt the same localise-repair-validate loop.
- 02 Code Generation Benchmarks and pass@k How HumanEval turned code evaluation into execution, why the unbiased pass@k estimator exists and how it is derived, and why saturation and contamination pushed the field toward time-stamped benchmarks.
- 03 Code Review by Model Why review is a better fit for current models than authoring, the precision problem that determines whether it is used, and the classes of finding worth reporting.
- 04 Fill-in-the-Middle and Completion Models Why a left-to-right model cannot do the most common editing task, the training transformation that fixes it, and the latency budget that shapes everything about inline completion.
- 05 Repository Context and the Retrieval Problem Why a model with a large context window still cannot see a codebase, what makes code retrieval different from document retrieval, and the signals that actually locate relevant code.
- 06 SWE-bench and Agentic Coding Evaluation What resolving a real GitHub issue measures that a function-completion benchmark does not, the contamination and scoping problems that complicate it, and how to read a reported score.
- 07 Test-Driven Agents and Verification Loops Why an executable check transforms an agent's reliability, how to structure the loop so it converges, and the failure where the agent optimises the test rather than the code.