A commerce platform can tell that its peak-day checkout success rate has dropped, but cannot tell which of forty services is responsible. What is missing, and in what order should it be added?

What is missing

Attribution. The platform has monitoring — it detected the symptom — but not observability, which is the ability to ask new questions of the system without shipping new code.

The specific gap is that each service reports its own health, and every one of them looks fine. A service that is fine 99.5% of the time is a healthy service and a broken checkout when it sits on a path of forty of them.

The order to add things

1. Correlation identifiers, propagated everywhere. Without a request id flowing through every hop — including asynchronous work and background jobs — no other tool can connect the symptom to a cause. This is cheap, it is a prerequisite for everything else, and retrofitting it is the single most valuable observability investment most platforms can make.

2. Distributed tracing on the critical path. Sampled, but with a rule that guarantees traces for errors and slow requests, because uniformly random sampling misses exactly the requests you need. A trace turns "checkout is slow" into "checkout is slow because the tax service p99 tripled".

3. RED metrics per service and per endpoint — rate, errors, duration — with consistent naming, so comparison across services is possible at all.

4. Business metrics next to technical ones. Checkout success rate, orders per minute, payment authorisation rate. These are the SLIs that matter, and they are frequently the only signal that catches a failure with no technical error — a payment provider silently declining, or a bug that produces a valid response containing wrong data.

5. Exemplars linking metrics to traces, so a spike on a dashboard is one click from an example of the slow request. This is what turns a dashboard from a status display into an investigation tool.

The peak-day specific requirement

Segmentation by whatever concentrates load. On a commerce platform that is the largest merchants: a handful of shops generate a disproportionate share of traffic, and a failure affecting only them is invisible in an aggregate and catastrophic commercially.

An aggregate success rate cannot distinguish "everyone slightly degraded" from "our biggest customers completely broken", and those require entirely different responses.

The principle

Monitoring tells you something is wrong; observability lets you find out what, without deploying. The distinguishing test is whether you can answer a question nobody anticipated — and that capability is built out of high-cardinality context attached to requests, which is exactly what dashboards of pre-aggregated counters cannot provide.