A new service goes live next week. What observability must exist on day one?
Show the full answer Hide the answer
What the interviewer is testing
Whether you can name a minimum viable set rather than an aspirational platform.
The day-one minimum
The four signals per endpoint: request rate, error rate, duration distribution (p50, p95, p99), and saturation of the constrained resource — usually the connection pool or thread pool rather than CPU.
Structured logs with a correlation identifier propagated from the edge, so one request can be followed across services. The identifier must be consistent with the rest of the estate, or the service is an island.
Distributed tracing instrumented at least at the boundaries — inbound, outbound calls, database queries. This is what makes a slow request diagnosable rather than a guess.
Health checks that exercise the real dependency path, not a constant response.
One dashboard that answers "is it working" in five seconds, and links to the traces.
Alerts on symptoms, not causes: error rate and latency against a target, plus at least one check that the critical journey works end to end. Not CPU, not disk, not queue depth — those are dashboards.
A runbook naming the owner, the escalation path, the dependencies, and what to do for the two or three most likely failures.
What can wait
Profiling, business metrics dashboards, anomaly detection, log-based analytics, and sophisticated SLO tooling. Useful; not what determines whether the first incident is survivable.
The question that sets the bar
"When this fails at 3 AM, how will the team know what happened?" Walk through it concretely for the likeliest failure — a slow dependency — and check that each step is answerable with what exists.
What a strong answer adds
That this list should not be assembled per service. It belongs in the service template and the platform, so a new service inherits correct telemetry, a dashboard and alert routing on day one without anyone deciding to build them. Observability configured per team at launch is observability that varies in quality and is missing when it is needed.
Common weak answers
Logs alone. A comprehensive observability strategy that will not exist by next week.