Compute Economics
Capex versus tokens, utilisation and depreciation, price-performance curves and the cost floor of inference.
5concepts
56flashcards
35minutes of reading
- 01 Concentration, Supply and the Compute Market Why AI compute has an unusual supply structure, what the constraints actually are at each layer, and how that shapes strategy for organisations that only want to buy some.
- 02 Training Versus Inference Spend Why inference dominates the lifetime bill for any successful model, the crossover arithmetic, and how that changes which optimisations are worth doing.
- 03 Capex, Depreciation and the Cost of a GPU-Hour How a purchased accelerator's cost becomes an hourly rate, why the depreciation schedule is the contested assumption, and what utilisation does to the answer.
- 04 The Cost Floor of Inference What sets the minimum achievable cost per token, why memory bandwidth rather than compute is the binding constraint, and which techniques move the floor rather than approaching it.
- 05 The Price-Performance Curve Why cost per unit of AI capability has fallen far faster than hardware improvement alone, the three compounding contributions, and what that implies for planning.