advanced 3 min answer

A colocation contract prices space by provisioned power per rack rather than by server count. A proposal replaces 200 general-purpose servers with 40 accelerator servers - a fivefold cut in machine count - while tripling the power each rack draws. What does the team gain, what does it pay, and when does that bill arrive?

tcocolocationpowerdensitycapacity-planningcooling
Show the full answer Hide the answer

What is gained

Fewer machines to patch, monitor and replace. Better performance per watt on the work the accelerators are built for. And if any software in the stack is licensed per socket or per core, a fivefold cut in hosts can cut the licence line by a similar factor, which is often larger than the hardware saving.

What is paid, and in what unit

In an owned or colocated model the billable unit is provisioned kilowatts, not rack units, and the rack has a power and cooling envelope that is fixed by the hall's design. Work it through. 40 nodes at 6 kW each is 240 kW. In a hall provisioned at 10 kW a rack, that is 24 racks holding 40 nodes: fewer than two machines a rack, with roughly 80% of the rack space bought and empty.

The machine count fell fivefold and the rack count barely moved, because the constraint was never floor space. Three further costs follow:

  • Stranded space, paid for at the rack rate while standing empty.
  • A contract commitment. Provisioned power is normally a committed figure, and increasing it has a lead time while decreasing it mid-term usually does not reduce the bill at all.
  • Cooling. A hall designed for low density may need containment or liquid cooling to remove 18 kW from a rack, and the facility's power usage effectiveness worsens as it works harder, so you pay for the overhead power too.

When the bill arrives

At the next capacity increment, not at this one. The hall runs out of provisioned power long before it runs out of floor, so the next ten accelerator nodes do not trigger a purchase order for ten machines; they trigger a power order with a lead time measured in months, and possibly a new hall. A team that planned in machine counts discovers this when the growth it promised is physically unavailable.

How to keep the option to reverse

Choose the unit before the vendor does: denominate the whole model in kilowatts and dollars per kilowatt-month, and convert to a per-machine figure only at the end for presentation. Contract power in increments with a stated ramp rather than one large commitment. Concentrate the dense workload in one hall so the retrofit is bounded, and keep a public-cloud path for burst, so a power shortfall degrades to a higher unit cost rather than to no capacity.

When this is the wrong answer

In public cloud none of this is visible, because you rent the machine and the provider absorbs the density problem. That absorption is one of the genuinely valuable and rarely-counted services inside the cloud premium, and a repatriation model that ignores it understates the owned side. If the comparison is cloud against colocation and the workload is dense, price the colocation side in kilowatts including the cooling retrofit, or the model will favour owning for reasons that evaporate on installation day.

The trade also reverses at low density. For ordinary serving fleets drawing a few hundred watts a node in production, floor space and machine count really are the binding constraints, and consolidating onto fewer larger machines is straightforwardly cheaper.