An internal platform hosts workloads from many teams. What isolation model should it use, and how are noisy neighbours handled?
Show the full answer Hide the answer
The isolation spectrum
Shared cluster, namespace per team. Cheapest, best utilisation, weakest isolation. Shared control plane, shared nodes, shared network — a team can affect others through resource consumption, control-plane load, or a node-level failure.
Shared cluster, dedicated node pools per tier or team. Compute isolation with a shared control plane. A reasonable middle position and where most platforms land.
Cluster per team or per environment. Strong isolation, poor utilisation, and multiplied operational burden — the upgrade problem alone becomes substantial at dozens of clusters.
Cluster per tenant with full separation, for genuinely hostile multi-tenancy or hard compliance boundaries.
Choosing
Start from the threat model rather than from a preference. Internal teams that trust one another need protection from accidents, not from attacks — which is a much cheaper problem, and it is where most internal platforms actually sit.
The questions that decide it: is a compliance boundary required between workloads; do any workloads run untrusted code; what is the consequence of one team's workload degrading another's; and can the platform team operate the number of clusters the strong-isolation option implies? That last question decides more real cases than the others.
Handling noisy neighbours
- Resource requests and limits on everything, enforced by admission policy. A workload with no request is scheduled as though it needs nothing, which is the root of most noisy-neighbour incidents.
- Understand the difference between CPU and memory limits. CPU limits throttle — and CPU throttling produces latency spikes that are invisible in average utilisation and are a common cause of unexplained tail latency. Memory limits kill. Setting a CPU limit equal to the request is a frequent and damaging default, because it prevents a workload from using idle capacity.
- Namespace quotas capping a team's total consumption.
- Priority classes with preemption, so critical workloads displace batch work under pressure.
- Node pools by workload class — latency-sensitive serving separated from batch and from anything with unpredictable resource behaviour.
- Control-plane protection: API rate limits per tenant, since a team's controller in a hot loop can degrade the API server for everyone, which is a real and frequently-overlooked shared resource.
- Storage and network isolation, which are shared resources people forget: a workload saturating node network bandwidth or local disk affects its neighbours regardless of CPU and memory limits.
The failure most platforms hit
The platform's own components are unisolated. Ingress controllers, log shippers, metrics agents, service mesh sidecars and CSI drivers run on every node, and a tenant workload that starves them takes down the platform's ability to observe and route — while every tenant-facing quota is being respected.
Platform components need their own reservations, priority classes and, ideally, their own node pool, and this is consistently discovered during the first serious incident rather than during design.