A new Java service takes about 90 seconds to load its caches and become useful. After deployment every pod restarts every minute or so and never serves traffic. The liveness probe starts at 5 seconds and allows three failures at 10-second intervals. What is the cleanest fix?

The deciding property

Liveness answers "is this process wedged and worth killing", and during boot the honest answer is "it is not wedged, it is not finished". With the settings in the stem the liveness probe begins at 5 s and gives up after roughly 5 + 3 x 10 = 35 s. The service needs 90 s. The kubelet restarts it at second 35, and the restart resets the clock, so no pod ever reaches the point where it would pass. It is a loop created by the probe, not by the application.

Why the startup probe is the right tool

A startup probe runs first and, until it succeeds, disables liveness and readiness checks entirely. Give it a generous budget, for example a failure threshold of 30 at a 5-second period, which allows 150 s to boot. Once it passes, the strict liveness probe takes over with its normal tight thresholds of 20 to 30 s, so a genuine deadlock later is still caught quickly. You get slow-start tolerance and fast deadlock detection together, and each number answers one question.

Why the other options fail

• "Raise the liveness failure threshold to 30." It stops the loop, and it is what teams did before startup probes existed. The price is permanent: a pod that deadlocks at 3 am now takes five minutes to be restarted, because the same setting governs steady state. Right only if the platform predates startup probes.
• "Switch the liveness probe to check the database connection." This is worse than the original problem. A liveness check that depends on a shared system turns a slow database into a fleet-wide restart storm, and it does nothing about the 90-second boot.
• "Remove the readiness probe." Readiness decides whether traffic is routed to the pod; it does not restart anything. Removing it sends requests to a half-warm instance and leaves the restart loop intact.

Common weak answers

• "Just increase initialDelaySeconds." It is a fixed guess; the day boot takes longer the loop returns, and a fast boot waits for nothing.

What it costs and when it flips

A startup probe adds one more setting to keep honest: if boot time doubles to 200 s after a release, the pod is killed at the end of the startup budget and the loop reappears. Alert on boot duration against the budget, and keep the budget at roughly twice the measured p99 boot.

If boot takes 3 seconds, skip the startup probe and set a small initialDelaySeconds; the extra object is not worth carrying. If boot is slow because of work that could happen after the port opens, such as warming a cache, serve a degraded response and let readiness flip when ready, which removes the long boot rather than tolerating it.