Marginal Request Cost
also called Incremental Cost per Request, Next-Request Cost
The cost of serving one more request given current capacity, which is near zero inside a capacity step and a whole step at its boundary - so it is the wrong number to price a contract with and the right one for a capacity decision.
A sales team asks whether the platform can absorb a customer who adds 20% more traffic, and at what cost. Finance answers with the average: total spend divided by requests, say $0.004 a request, so 20% more traffic costs 20% more money.
That answer is wrong in both directions depending on the day it is given. If the fleet runs at 60% of its current nodes' capacity, those requests cost almost nothing. If it runs at 95%, the same requests buy a node, a shard, a licence block or the next commitment tier, and the marginal cost of that increment is a whole step. Average cost tells you whether the business is viable; marginal cost tells you what the next decision costs.
Why it matters
Infrastructure cost is a staircase, not a line. Nodes, shards, replicas, licence blocks, provisioned throughput units and support plans arrive in indivisible increments. Inside a step the marginal cost is only the variable meters: transfer, per-request charges, third-party per-transaction fees. At a boundary it jumps.
That matters in the two places it is most often ignored. Deal pricing, where the average is right because the fixed floor must be recovered over a contract that outlives the headroom. And efficiency work, where the marginal figure is right, because optimising inside a step with plenty of headroom saves nothing until the saving removes a step.
Implementation patterns
- Split the model into fixed-per-step and variable-per-request terms. The variable terms are marginal cost inside a step; step size and current headroom give the boundary.
- Publish headroom beside unit cost. "$0.004 average, $0.0004 marginal, 38% headroom to the next node" answers three questions one number cannot.
- Express efficiency wins in steps removed, not percentages saved. A 15% CPU reduction that removes no nodes saves nothing this month.
- Separate third-party per-transaction fees. These are genuinely marginal and unavoidable, and they put a floor under marginal cost that no infrastructure work can lower.
Industry example
Committed-volume delivery contracts make the staircase visible. A platform commits to a monthly volume at a discounted per-gigabyte rate, so inside the commitment the marginal gigabyte is already paid for and marginal cost is zero until the commitment is exhausted, after which overage rates apply and marginal cost rises above the average. A team pricing incremental traffic at the blended average refuses free volume for most of the month and then underprices it at exactly the point it becomes expensive.
The same shape has been visible in production estates for as long as reserved pricing has existed: zero up to the reserved quantity and on-demand above it, zero up to the licensed core count and then a whole block, zero up to provisioned throughput and then throttling.
Failure scenarios
- Refusing profitable volume. A deal declined on average cost while the fleet has 40% headroom, so the increment would have been nearly all margin.
- Underpricing at the boundary. The same deal signed when the fleet is saturated, buying three nodes, a shard rebalance and a larger commitment.
- Marginal cost used for pricing. The contract covers variable cost only, the fixed floor goes unrecovered, and margin falls as volume grows.
Trade-offs
| Choose | Gains | Pays |
|---|---|---|
| Price on average cost | The fixed floor is recovered and margin is stable | Free capacity goes unsold while headroom exists |
| Decide capacity on marginal cost | Investment goes where it removes a step | A number valid only until the next step |
When not to use it
In a fully usage-metered or serverless architecture the staircase mostly disappears, because the provider has absorbed the steps into a per-invocation price; average and marginal converge and one number is honest. The same holds for an estate whose single step is the whole system: with one database and three containers, the marginal cost of anything is either zero or everything.
Interview question
Q: Your average infrastructure cost is $0.004 per request. A prospect would add 20% to your traffic on a three-year term. What do you need to know before quoting, and which number do you quote?
What a strong answer covers: headroom to the next step in every tier, not only compute; which terms are genuinely variable, including third-party fees; that the quote must use average cost because the contract outlives the headroom; and that the marginal figure decides whether to onboard this quarter or next. The strongest answers note that a commitment crossing several steps needs a step schedule rather than a unit rate.
Quick check
Quiz: Your fleet runs at 95% of provisioned capacity and a change cuts CPU per request by 10%. What is the saving this month? Likely zero, because the nodes are already paid for; its value is the deferral of the next capacity step, which is how it should be presented.
Flashcard: Which cost per request prices a contract, and which decides capacity? — Price on average, because the fixed floor must be recovered over the contract's life. Decide capacity on marginal, because that is what the next request really costs until the step boundary.