Inference & Serving
21 min
Cache-Augmented Generation: When Preloaded KV-Caches Replace Retrieval Pipelines
Retrieval-augmented generation fetches documents at query time, scores them, and hopes the retriever got it right. Cache-Augmented Generation sidesteps the entire pipeline by preloading knowledge into the model's KV-cache before …