concept

Feedback Loop

A circular causal chain in which an effect influences its own cause, producing behaviour that cannot be understood by examining components in isolation.

The core systems-thinking construct, and the one that explains most surprising production behaviour.

Reinforcing loops amplify. Requests time out, clients retry, load increases, more requests time out. Nothing in the system is behaving incorrectly and the outcome is collapse. Retry storms, cache stampedes and death spirals are all this shape.

Balancing loops stabilise — autoscaling, circuit breakers, backpressure, rate limiting — and each introduces its own dynamics. A balancing loop with too much delay oscillates: an autoscaler responding to a five-minute average adds capacity after the spike and removes it before the next one.

Three practical consequences:

Delay in a loop is as important as its strength. Most oscillation in production systems comes from lag between signal and response, not from the wrong response.

Local optimisation can worsen the whole. A team that improves its own latency by retrying aggressively degrades the shared dependency for everyone.

Individually correct behaviours can combine into failure. Meta's 2021 outage came from name servers correctly withdrawing routes when isolated — every component behaving as designed, the system removing itself from the internet.

The discipline this suggests: when something surprising happens, look for the loop before looking for the faulty component.