Service Discovery in Practice

Definition

The mechanism by which a caller determines the network location of a healthy instance of a service, in an environment where instances are created, destroyed and rescheduled continuously.

Why it matters

Static configuration cannot work when instances are ephemeral. A container rescheduled onto a different node has a different address, and autoscaling changes the set continuously.

The subtler point is that discovery is not only address resolution — it is health-aware address resolution. Returning an instance that exists but is not serving is worse than returning nothing.

Implementation patterns

Client-side discovery. The client queries a registry and chooses an instance itself. Efficient — no extra hop — and it puts load-balancing logic in every client, in every language.

Server-side discovery. A load balancer or proxy resolves and routes. One place to implement the logic, one more hop, and the balancer must be highly available.

Sidecar / mesh. A local proxy handles it, giving the client-side model without embedding logic in each application. This is a large part of what a service mesh buys.

DNS-based. Universally supported, and constrained by caching behaviour: resolvers and application runtimes cache aggressively and frequently ignore TTLs, which makes DNS a poor mechanism for fast failover.

Platform-native, where the orchestrator provides it — the simplest option when available and the one to prefer.

Failure scenarios

The registry becomes a hard dependency of the data plane. If a service cannot serve requests when discovery is unavailable, the registry's availability is now an upper bound on everything. The remedy is caching the last known good topology and continuing to use it, degrading rather than failing.

Health check quality. A shallow check that returns a constant marks a gray-failing instance healthy; a deep check that exercises every dependency causes cascading removal when a shared dependency degrades. The usual answer is a shallow liveness check and a deeper readiness check.

Registration and deregistration races, where an instance receives traffic before it is ready or continues receiving it after termination begins. Both are solved by readiness gating and connection draining.

Stale entries for instances that died without deregistering, requiring the registry to expire entries on missed heartbeats.

Industry example

Roblox's 2021 outage is the cautionary one. Their Consul cluster provided service discovery and configuration for essentially every service, and when it degraded under a novel load pattern the whole platform was down for roughly 73 hours. The centralisation that makes discovery valuable is exactly what makes it a shared point of failure — and the mitigation is local caching with a long fallback so the data plane survives a control-plane outage.

Trade-offs

Client-side is efficient and duplicates logic. Server-side centralises and adds a hop. Mesh gives both benefits and adds a control plane with its own upgrade cycle and failure modes.

Strong consistency in the registry gives a correct view and makes availability harder; eventual consistency is more available and means callers occasionally try a dead instance — which is fine if they retry.

Interview question

What happens to your services when the service registry becomes unavailable?

The answer that matters is whether the data plane continues. Strong candidates describe caching the last known good endpoint set with a long fallback, and identify the general principle: do not put a control plane in the data plane's request path.