Load Balancing Algorithm
The rule deciding which backend receives a request — round robin, least connections, least response time, or hash-based.
Round robin distributes evenly by count and ignores whether a backend is struggling. Fine when requests are uniform and backends identical; poor when either varies.
Least connections sends to the backend with fewest in flight, which naturally routes away from a slow instance — usually the better default for variable request costs.
Least response time goes further, weighting by observed latency. Most responsive to grey failure, and it can oscillate if not damped.
Hash-based (by client IP, session, or a key) gives consistent routing, which is required for sticky sessions and useful for cache locality. Costs even distribution.
Power of two choices — pick two at random, send to the less loaded — gets most of the benefit of least-connections without the coordination cost, and is what several modern proxies use.