The Price-Performance Curve
Why cost per unit of AI capability has fallen far faster than hardware improvement alone, the three compounding contributions, and what that implies for planning.
The cost of reaching a given AI capability has fallen much faster than any single technology curve would explain. Understanding which contributions compound is what separates a defensible planning assumption from an extrapolation of a trend nobody has decomposed.
Three multiplying factors
Hardware. Each accelerator generation delivers more throughput per dollar and per watt, through more compute, more memory bandwidth, and support for lower-precision formats. The last is not a small effect: moving from fp16 to fp8 to fp4 arithmetic multiplies effective throughput at each step for workloads that tolerate it.
Software and systems. Better kernels, better parallelism strategies, higher achieved utilisation, better serving. FlashAttention, paged KV cache management, continuous batching, speculative decoding and prefix caching each delivered substantial improvements on unchanged hardware, and they compose.
Models and algorithms. Better architectures, better training recipes and better data mean fewer parameters and fewer tokens are needed for a given capability. Mixture of experts decoupled capability from active compute; distillation moved capability into smaller models; improved data quality moved the scaling curve rather than the position on it.
Multiplied together these produce a decline in cost per unit of capability far steeper than hardware alone, and observed API price reductions for a fixed capability level over the past few years have been dramatic, though the specific multiple depends heavily on which capability and which model family is being tracked.
What this means for planning
Cost estimates decay quickly. A unit economic model built on today's prices should be revisited on a schedule, and a business case that only works at today's prices may work comfortably in a year or may be irrelevant.
Long commitments forgo the decline. A three-year commitment at today's price-performance is a bet that the decline will be slower than the discount. Shorter commitments cost more per hour and retain the option.
Capability targets move too. The frontier advances, so the capability that is expensive today may be table stakes later, which affects both what to build and what to differentiate on.
Absolute spend rises anyway. Falling unit costs have historically increased total consumption rather than reducing total spend, because cheaper capability makes more applications viable. Planning for a lower bill because unit prices fell is a common and reliable disappointment.
When it breaks
Extrapolation is not guaranteed. The gains came from specific advances, and there is no law requiring more. Process node improvements have slowed, and the algorithmic contributions are lumpy rather than smooth. A planning assumption of continued decline should be stated as an assumption.
Price is not cost. Provider pricing reflects competition, strategy and capacity as well as cost, so an observed price decline may not reflect a cost decline, and it can reverse when capacity tightens.
Improvements are not uniform. Long-context inference, reasoning models that emit many tokens, and agentic workloads with many calls have cost profiles that have improved less than simple single-turn generation. A blended average hides which workload is getting cheaper.
Cheaper per token can be more expensive per task. A reasoning model at a lower per-token price that emits ten times as many tokens costs more per answer. Comparing on price per token rather than price per completed task is the error this most often produces.
12 flashcards for this concept
Click a card to reveal the answer.