Bin-Packing Efficiency

A platform team spends a quarter reducing container requests across 900 deployments. Average requests fall 40%, the tooling reports a large saving, and the node bill does not move. Nothing is broken and nothing was saved, because the organisation buys nodes and the campaign optimised requests.

Bin-packing efficiency is the ratio that connects the two: the sum of scheduled pod requests divided by the sum of node allocatable capacity, computed per node pool. Capacity freed inside a node that nobody removes is headroom, not money.

Why it matters

Cloud compute is sold in fixed shapes. A scheduler places variable-shaped workloads into those shapes, and the remainder is paid for and unused. At 55% packing efficiency, 45% of the compute line is stranded - not idle in the sense of a low-utilisation instance, but reserved capacity that no workload has even claimed.

The metric also identifies the binding dimension. A pool can be full on memory at 90% while CPU sits at 25%, and a CPU-focused rightsizing exercise on that pool cannot remove a single node. Publishing efficiency per resource per pool turns a vague efficiency programme into one arithmetic question: which resource runs out first, and what is stopping the emptied nodes from being deleted?

Implementation patterns

• Compute it per node pool and per resource, never as a cluster average. Mixed pools average away the pool that is actually wasting money.
• Subtract the per-node tax first. DaemonSets, the kubelet reservation and system overhead consume a fixed slice of every node, so allocatable rather than capacity is the denominator, and the tax gets proportionally worse as nodes get smaller.
• Track the pod-count ceiling as a third resource. Many managed distributions default near 110 pods per node, and a fleet of small sidecar-heavy pods hits that limit while CPU and memory sit half empty.
• Alert on stranded capacity, not on pod utilisation: requested-to-allocatable below roughly 60% for a pool sustained over 24 hours is a finding worth someone's morning.
• Fix the scheduling pins before resizing anything: a PodDisruptionBudget at zero unavailable, a pod with local storage, or an un-annotated DaemonSet each pin a node permanently, because a node is removed only when every pod on it can be rescheduled.
• Match node shape to pod shape last and deliberately. This is usually the largest single saving and it requires draining the fleet, which is why teams skip it.

Industry example

Managed Kubernetes providers have shipped consolidating node provisioners since roughly 2021, precisely because request-level tuning does not reach the invoice: these controllers actively repack workloads and delete under-used nodes rather than waiting for one to become empty. The design admits the problem the metric names. The same arithmetic governs every scheduler that buys fixed-size capacity, from a container platform to a Hadoop-era YARN cluster.

Failure scenarios

• The silent quarter. Requests drop, efficiency rises, node count does not, and nobody reconciles the two until finance asks about the promised saving.
• The shrinking-node trap. A team moves to smaller instances to improve packing, and the fixed per-node DaemonSet tax now consumes a larger share of each node, so efficiency falls.
• The anti-affinity floor. Topology spread rules hold nodes open that packing alone would have emptied, and the constraint is invisible in any cost report.
• Efficiency optimised into an incident. Packing to 95% removes the spare capacity that absorbs a node failure, so the next lost node cannot be rescheduled and pods sit pending.

Trade-offs

The honest position is that some unpacked capacity is insurance and should be named as such, with a target rather than an apology, so that the next efficiency campaign does not remove it by accident.

When not to use it

On a fleet of a few dozen nodes the metric is true and not worth acting on: the engineering time to unpick scheduling pins exceeds the saving, and buying a smaller node shape captures most of it. It is also the wrong metric for serverless and per-request billing models, where there is no node to pack and the unit of waste is concurrency rather than capacity. And on a single-tenant pool running one large workload, packing is a property of that workload's shape, not of the scheduler.

Interview question

Q: Your rightsizing programme cut average pod requests by 40% across the estate and the compute bill is flat. Walk me through how you would find where the saving went, and what you would change so the next campaign reaches the invoice.

What a strong answer covers: the requests-versus-nodes distinction stated immediately; bin-packing efficiency per pool as the diagnostic; identifying the binding resource, including pod count; the scale-down preconditions and the specific pins that block them; node shape as the largest and last lever; and a target that deliberately leaves headroom for failure rather than packing to the ceiling.

Quick check

Quiz: Requests fell 40% and node count did not change. What is the one ratio that explains it? — Bin-packing efficiency: requested capacity over allocatable capacity per pool. You pay for nodes, and freed space inside a node saves nothing until the node is deleted.

Flashcard: Why can a 40% cut in container requests produce a 0% cut in the bill? — Because nodes are the billed unit. Scale-down happens only when every pod on a node can be rescheduled elsewhere, and PDBs, local storage, DaemonSets and anti-affinity routinely prevent that.