A core internal service has run continuously for three years. What risk does that create?
Show the full answer Hide the answer
What the interviewer is testing
Whether you recognise that untested recovery paths are latent outages, and that uptime can be a risk signal rather than a health signal.
The risk
Its cold-start behaviour is unknown. Three years of continuous operation means the startup path has not been exercised at current scale, with current data volume, current dependencies and current configuration.
Specific unknowns: how long a full restart takes, whether it can even start given how much state has accumulated, whether its startup dependencies still exist and still behave as expected, and whether the configuration in code matches the configuration actually running after three years of drift.
Recovery time is therefore not a measured property, and any recovery objective referencing it is fiction.
Why this is not hypothetical
AWS's 2017 S3 incident is precisely this. A mistyped command removed more capacity than intended, requiring a full restart of subsystems that had not been fully restarted in years — and the restart and its safety checks took far longer than anyone expected. The removal was the trigger; the untested restart path was what made it a multi-hour outage.
What to do
Restart it deliberately, at a low-traffic time, with a timer running and the team present. This is the entire remedy and it costs an afternoon.
Then reduce the restart time if it is unacceptable: partition the state so recovery is parallelisable, add snapshots so replay is bounded, or provision so that a rolling restart never takes everything down at once.
Make it routine. A service that restarts weekly — through deployment, node rotation or deliberate recycling — has a continuously exercised startup path. Immutable infrastructure gives this as a side effect, which is one of its underrated benefits.
Check the accumulated drift while you are there: does the running configuration match the code, and would a rebuild from source produce what is currently running?
What a strong answer adds
Reframing the metric. Long uptime on a stateful service is a risk indicator, not an achievement — it means an untested recovery path, unpatched underlying systems and accumulated drift.
And the general question for any critical component: when did this last cold start, and how long did it take?
Common weak answers
Treating three years of uptime as evidence of reliability. Adding monitoring, which does not test the restart.