Search Indexing Service  ·  View 17 of 21  ·  Operations

Observability

Signal type by pipeline stage — and one row that exists because the others cannot see this platform's worst failure.

Editable source SVG draw.io All views
Capture Log Assembly Index write Query Lifecycle Lag & freshness Source commit → log Partition lag Consumer lag per lane Bulk queue depth Observed freshness served Catch-up lag before swap Correctness Rejected / quarantined Dup + out-of-order rate Degraded-enrichment share Unmapped-field errors Partial-result rate Divergence rate the alarm that matters Latency & load Accept p99 Produce p99 Fan-out backlog Indexing throughput Query p50 / p95 / p99 Rebuild docs/s Silence detectors Source quiet-period alarm no signal is a signal Empty partition alarm Schema-drift alarm Rehearsal not run Cost Enrichment cost share Cost per M events Cost per M queries Query : write ratio Observability — Signal by Pipeline Stage The silence row exists because this platform's worst failure — a document that never arrived — produces no error anywhere else on this grid. v 1.0 · owner Data Platform Architecture

The silence row

  • A source that stops emitting, a partition that goes empty, a drift that becomes a dropped field, and a rebuild rehearsal that was never run all produce no error anywhere else on this grid.
  • Each is therefore an alarm on an absence: a declared quiet period per source binding, not a threshold on a rate.
  • The divergence rate from continuous reconciliation is the single number that says whether the index still matches the world.

Decisions

  • Freshness is measured end to end per lane, from source commit to searchable, not as consumer lag — because consumer lag can be zero while the index is wrong.
  • Every response carries the index version and freshness it was served at, which makes a staleness complaint reproducible.
  • The query-to-write cost ratio per index is published to tenants, because an index maintained more eagerly than it is read is the cheapest saving available.