A platform's health check returns 200 while the service cannot serve real traffic. What should a health check actually verify, and what should it deliberately not?

The three distinct questions

• Liveness: is this process functioning, or should it be restarted? Must be shallow and must not depend on anything external — a liveness check that fails because a database is down causes every instance to be restarted during a database outage, turning a dependency problem into a total outage.
• Readiness: should this instance receive traffic right now? May consider local state — warm caches, established connection pools, completed startup — and may consider critical dependencies, carefully.
• Deep health: is the whole system working end to end? Belongs in monitoring and synthetic checks, not in a load balancer's decision.

Conflating them is the usual failure, in both directions: a check too shallow keeps a broken instance in rotation; a check too deep removes every healthy instance when a shared dependency has a blip.

What readiness should verify

The instance's own ability to serve: startup complete, configuration loaded, connection pools established, caches warm enough to avoid a cold-start stampede.

The dependency question is the hard one. If a service is useless without its database, then depending on it seems right — but if every instance checks the same database, a brief database issue removes the entire fleet simultaneously and prevents the recovery that would have happened as connections re-established.

The safer pattern is to fail requests rather than fail readiness, so the instance stays in rotation, returns errors that are visible in the SLI, and recovers immediately when the dependency does. Failing readiness should be reserved for problems local to that instance.

What makes a health check useful rather than decorative

• It exercises the real path, at least shallowly — a static 200 handler proves the process is running and nothing else.
• It has a timeout shorter than the checker's interval, or a slow check produces flapping.
• It is cheap, since it runs constantly from many checkers.
• It reports why, so an operator can distinguish "still warming" from "dependency unavailable".

The failure this prevents

The classic incident is a deep readiness check plus a dependency blip removing an entire fleet from rotation at once, at which point there is no capacity to serve the traffic that the dependency's recovery would have allowed. The health check caused a larger outage than the fault it detected — which is the specific reason to keep liveness shallow and readiness local.