Subnetting and Address Planning
Dividing address space across zones and tiers, and the exhaustion failures that appear only at container density.
Definition
A subnet is a slice of a VPC's address range, placed in one availability zone. Instances, containers and managed service endpoints all consume addresses from it.
The planning rules that prevent later pain
- Subnet per tier per availability zone. Three zones and three tiers is nine subnets. This is the standard layout because a subnet does not span zones, and high availability requires resources in each.
- Reserve room per subnet. Providers reserve several addresses in every subnet for their own use, so a /28 gives fewer usable addresses than arithmetic suggests.
- Size for the density you will have, not the density you have. This is where most planning fails: a subnet sized for 200 VMs is exhausted by a container platform where every pod consumes an address, and pod counts are an order of magnitude higher than instance counts.
- Leave contiguous gaps between allocations, so a subnet can be grown or a new one added adjacently rather than fragmenting the space.
- Document the allocation scheme — which range belongs to which environment, region and tier — and keep it somewhere findable. This document is what prevents the overlap that blocks a future peering.
The container-density failure
The most common modern address exhaustion story: a cluster grows, pods stop scheduling, and the error is opaque. The subnet has no free addresses, but nothing says so in terms an application team recognises. It presents as a capacity problem and is an address planning problem.
The mitigations are real architecture decisions: a secondary CIDR range for pod addresses, an overlay network so pods do not consume VPC addresses at all (at the cost of some observability and direct reachability), or larger subnets from the start. The last is free if done early and expensive afterwards.
Failure scenarios
- Pods unschedulable due to address exhaustion, diagnosed as a compute capacity issue.
- Subnets sized identically regardless of tier, so the application tier exhausts while the database tier uses six addresses.
- A managed service consuming addresses unexpectedly — endpoints, load balancers and NAT all take from the same pool.
- No documented plan, so a new environment is allocated a range that collides with an existing one.
Interview question
"Pods have stopped scheduling on a cluster with plenty of CPU and memory available. What do you check?"