concept

Distributed Trace

A causally-linked record of one request's path across services, composed of spans that carry timing, attributes and parent relationships.

A trace answers the question metrics and logs cannot: where did this specific request spend its time, and what did it touch?

Structure: a trace ID identifies the request; each unit of work is a span with a start, a duration, attributes and a parent span ID. The parent links form the causal tree.

Context propagation is the whole game. The trace context must pass across every boundary — HTTP headers, gRPC metadata, message headers — and one service that fails to propagate breaks the trace into disconnected fragments. This is why tracing is adopted at platform level rather than service by service: partial adoption produces partial traces, which are far less than proportionally useful.

Asynchronous boundaries are the hard part. A message consumed later is causally linked but not temporally nested, which is what span links exist to express.

What traces are best at, in order: finding the slow hop in a chain, revealing the actual dependency graph (which reliably differs from the documented one), showing N+1 patterns as a visual signature, and explaining tail latency — where a p99 trace shows a specific cause that an averaged metric cannot.