An organisation finds most incidents are caused not by individual service failures but by unexpected interactions between healthy services. What does systems thinking add here?

What systems thinking adds

Attention to interactions and feedback loops rather than to components. Every service is healthy by its own measures and the system is broken, which means the problem is in the relationships.

The specific structures worth looking for:

1. Reinforcing feedback loops. Latency rises → clients time out → clients retry → load increases → latency rises. This is metastable: it persists after the original trigger is gone, so fixing the cause does not restore service. The only thing that works is shedding load, which feels wrong because the system looks like it needs more capacity.

2. Hidden coupling. Services that share a dependency, a connection pool, a cache or a control plane are correlated even though the diagram shows them as independent. Shared fate is invisible in a component diagram and visible in a deployment one.

3. Local optimisation with global cost. Each service's retry policy is locally rational and the aggregate is amplification. Each team's utilisation target is locally efficient and the aggregate is a queue everywhere. This is why retry policy belongs to the platform rather than to each team.

4. Delays in the loop. Autoscaling responds minutes after the signal, so it can overshoot and oscillate. Any control loop with a delay longer than the disturbance is capable of making things worse.

What to do about it

• Map dependencies including the implicit ones — identity, DNS, configuration, deployment, observability — which are absent from most diagrams and present in most outages.
• Look for amplification in retries, reconnects and fan-out, and cap it centrally with budgets rather than per-request counts.
• Inject faults deliberately to discover interactions, since the highest-value findings are almost always "this dependency we thought was optional is not" and "this fallback does not work".
• Instrument the loops, not just the components: queue depth over time, retry rate as a fraction of successes, reconnect rate.
• Review architecture for emergent risk, asking what happens when two things degrade simultaneously rather than one at a time.

The framing

A system's behaviour is not the sum of its components' behaviours. Component-level health checks, component-level SLOs and component-level testing can all pass while the system fails — and that gap is precisely where the expensive incidents live.