Compute Economics
Capex versus tokens, utilisation and depreciation, price-performance curves and the cost floor of inference.
6concepts
63flashcards
43minutes of reading
- 01 Concentration, Supply and the Compute Market Why AI compute has an unusual supply structure, what the constraints actually are at each layer, and how that shapes strategy for organisations that only want to buy some.
- 02 Energy and Power as the Binding Constraint Why a grid connection measured in megawatts, not a budget measured in dollars, now caps how much AI compute can be built, how PUE converts a power limit into an accelerator count, and why the demand forecasts disagree by a factor of two.
- 03 Training Versus Inference Spend Why inference dominates the lifetime bill for any successful model, the crossover arithmetic, and how that changes which optimisations are worth doing.