Target Tracking Scaling
An autoscaling policy that adds or removes capacity to hold a chosen metric near a target value, like a thermostat, rather than reacting to threshold breaches.
Simpler and better-behaved than step or simple scaling policies, because the target expresses intent directly: "keep average CPU near 60%" or "keep requests per instance near 200".
The important choice is which metric to track, and CPU is usually the wrong one. A service that is latency-bound — waiting on a database, a downstream API or a lock — sits at low CPU while its queue grows, so CPU-based scaling never triggers on the condition that is actually hurting users.
Better signals, roughly in order: requests in flight or concurrency per instance, which maps directly to Little's Law and to saturation; queue depth or oldest-message age for worker pools; requests per second per instance, if latency is stable. Cloud load balancers expose per-target request counts precisely so this is available.
Two configuration details that decide behaviour. The cooldown or warm-up period must exceed the time an instance needs to become useful, or the autoscaler adds more instances while the previous ones are still starting and then over-corrects. And scale-in should be slower than scale-out — aggressive scale-in removes capacity that a returning spike immediately needs, and the cost of being briefly over-provisioned is much lower than the cost of being under-provisioned.