LLM Application Architecture
Routing, fallbacks, caching layers, structured state, and the reference shapes production LLM apps take.
5concepts
60flashcards
35minutes of reading
- 01 Fallbacks, Timeouts and Degradation How to build an LLM feature that survives a provider outage, why the usual retry patterns need adjusting for generation, and what degrading gracefully looks like when the core capability is unavailable.
- 02 Model Routing and Cascades Why sending every request to the strongest model is usually wrong, the two routing patterns and what each requires, and the economics that decide whether routing pays.