Compute Economics
Capex versus tokens, utilisation and depreciation, price-performance curves and the cost floor of inference.
6concepts
63flashcards
43minutes of reading
- 01 Concentration, Supply and the Compute Market Why AI compute has an unusual supply structure, what the constraints actually are at each layer, and how that shapes strategy for organisations that only want to buy some.
- 02 Energy and Power as the Binding Constraint Why a grid connection measured in megawatts, not a budget measured in dollars, now caps how much AI compute can be built, how PUE converts a power limit into an accelerator count, and why the demand forecasts disagree by a factor of two.
- 03 Training Versus Inference Spend Why inference dominates the lifetime bill for any successful model, the crossover arithmetic, and how that changes which optimisations are worth doing.
- 04 Capex, Depreciation and the Cost of a GPU-Hour How a purchased accelerator's cost becomes an hourly rate, why the depreciation schedule is the contested assumption, and what utilisation does to the answer.
- 05 The Cost Floor of Inference What sets the minimum achievable cost per token, why memory bandwidth rather than compute is the binding constraint, and which techniques move the floor rather than approaching it.
- 06 The Price-Performance Curve Why cost per unit of AI capability has fallen far faster than hardware improvement alone, the three compounding contributions, and what that implies for planning.