Cost & FinOps for AI
Unit economics per request, token accounting, reserved versus spot capacity, and attributing spend to features.
12concepts
105flashcards
90minutes of reading
- 01 Cost per Completed Task Why a list price per million tokens cannot rank models for a workload, how to compute cost per completed task instead, and the measured evidence that the cheaper model per token is often the more expensive one per outcome.
- 02 Non-Token Charges in an AI Bill The line items a cost-per-token model cannot see, covering per-call tool fees, sandbox and session runtime, residency premiums and the token overhead of tool definitions, and why they decide the bill for agentic workloads.
- 03 Per-Request Energy, Water and Carbon How to put an energy, water and carbon figure on a single inference request, why the two most rigorous published per-request figures differ by a factor of 38 on carbon and 173 on water, and which accounting choices produce the gap.
- 04 Reserved, On-Demand and the Shape of Commitment How to choose a commitment mix when demand is uncertain, why the break-even is simply a price ratio, and the option value that makes shorter commitments rational despite costing more.
- 05 Self-Hosting Versus API Break-Even A worked break-even model for serving an open-weights model yourself against paying per token, showing why utilisation, not the hourly GPU price, usually decides the answer, and which assumptions flip it.
- 06 Spend Guardrails and Quotas Why AI spend can rise by orders of magnitude in hours, the layered controls that bound it without blocking legitimate work, and the design of a kill switch that is actually usable.
- 07 The Price Menu: Caching, Batch and Service Tiers The multipliers that sit between a model's list price and what you actually pay, the break-even arithmetic for a cache write, and why the menu exists as price discrimination rather than generosity.
- 08 Unit Economics of an AI Feature How to build a cost-per-request figure that survives scrutiny, why the marginal cost of an LLM feature does not fall with scale the way software's does, and the retry and failure multipliers everyone forgets.