Inference & Serving
4 min
Prompt caching is not just a cost optimisation - it changes what apps you can build
Most teams treat prompt caching as a billing hack. The deeper consequence is that you can now ship product patterns - persistent agents, large-context tools, system-prompt-heavy UX - that were latency-prohibitive eighteen months ago.