A compute platform must plan accelerator capacity where lead times are months, demand is uncertain, and jobs vary from minutes to weeks. How should the capacity model be built?

Why conventional capacity modelling fails here

Standard capacity planning assumes short lead times and reversible decisions: measure utilisation, project growth, add capacity when a threshold is crossed. With months of lead time, by the time the signal is clear the decision needed to be made two quarters ago.

Additionally, utilisation is a misleading signal for this workload. A cluster can report 95% utilisation while large jobs never start, because free capacity is fragmented into unusable pieces. The metric that looks healthiest coincides with the worst outcome.

What the model needs

1. Queue time as the primary demand signal, segmented by job size. "Large jobs wait three days; small jobs wait ten minutes" is actionable. Aggregate utilisation is not.

2. Demand modelled as a distribution, not a forecast. Given the uncertainty, the model should produce a range with probabilities, and capacity decisions should be evaluated against the range — including the cost of being wrong in each direction.

3. Asymmetric cost of error, made explicit. Under-provisioning costs delayed research and unusable capacity elsewhere; over-provisioning costs idle hardware on a depreciation schedule. These are rarely symmetric, and the asymmetry should drive the target rather than a utilisation number.

4. Fragmentation modelled explicitly. Effective capacity for large jobs is much lower than nominal capacity, and it depends on the mix of job sizes and on the scheduler's policy. The model must include the scheduler, or it will systematically overestimate available capacity.

5. Commitment structure as a decision variable. Reserved capacity, on-demand, spot and multi-provider allocations have different costs and different lead times. The plan is a portfolio, not a single number.

6. Workload shaping as an alternative to capacity. Checkpointing so jobs can be preempted, right-sizing requests, and shifting flexible work to off-peak periods all increase effective capacity without procurement — and are far faster to act on.

The scheduler's role in the model

Capacity and scheduling cannot be planned separately, because scheduling policy determines how much of the nominal capacity is usable:

• Gang scheduling prevents partial-allocation deadlock and reduces effective capacity slightly.
• Reservations with backfill recover most of that, allowing small jobs into gaps only if they finish before the reservation begins.
• Preemption with checkpointing turns low-priority capacity into a buffer for high-priority demand.

Modelling capacity without modelling these produces a number that does not describe the system.

The reporting discipline

Report queue time and job-completion throughput, not utilisation. Utilisation is what leadership asks for and it is actively misleading here — a cluster at 95% where large jobs never start is worth less than one at 80% where they do. The scarce resource is the completed job, not the occupied accelerator.