LLM Application Architecture
Routing, fallbacks, caching layers, structured state, and the reference shapes production LLM apps take.
5concepts
60flashcards
35minutes of reading
- 01 Caching Layers for LLM Applications The four distinct caches in a mature LLM system, what each requires to be correct, and why semantic caching is the one that is dangerous.
- 02 Structured Output and Constrained Generation Why parsing free text is the wrong integration point, how constrained decoding guarantees valid syntax, and what a guarantee about form does not give you about content.
- 03 Where State Lives in an LLM Application Why the context window is the worst place to keep state, the four stores a mature application actually uses, and the assembly step that decides what the model sees.