Leaderboard & Counting Service  ·  View 08 of 21  ·  Structure

Components and Boundaries

Two planes, and the rule that the request plane may read the processing plane's output but never wait on it.

Editable source SVG draw.io All views
Request plane — never waits on aggregation Write path Counting API Cloud Run Validator schema, bounds, clock Idempotency check 24 h horizon Hot-key detector shard assignment Abuse quarantine reversible hold Read path Query API Cloud Run Top-N resolver Rank resolver histogram percentile Own-write overlay read-your-writes Staleness tagger version + as-of Processing plane — rebuildable, restartable Streaming Aggregation job Dataflow, event time Accumulators exact, HLL++, min/max Shard fan-in Poison sink parse error kept Batch and scheduled Projection builder versioned output Histogram builder Season closer Workflows Rebuild runner ≥ 10× real time State High volume, rebuildable Event log Pub/Sub Event archive GCS, 90 d + 13 mo Bucket store Bigtable Ranked store Bigtable Rank cache Memorystore Low volume, exact Control plane store Spanner Closed standings immutable, 5 y Audit & retraction log Product backends Product clients Member directory top-N, rank ack after commit subscribe bucket deltas ranked view vN version-keyed read Leaderboard & Counting Service — Components and Boundaries Interface / broker Application we own Security / platform Risk / gap Queue / topic Data store External / third party synchronous event / async Two planes and one rule between them: the request plane may read processing-plane output but never waits on it. Product backends enter at the Counting API (view 09); the config API, the consoles, the warehouse export and every observability path are omitted for clarity. v 1.0 · owner Platform Architecture · date 2026-10

Decisions

  • Admission does four refusals before anything is logged — schema and bounds, idempotency, quota, and the inflation signal — because an event that should not count is cheapest to stop before it is durable.
  • The staleness tagger is a component, not a response header added by convention. Every read carries a projection version and an as-of, and that is enforced in one place.
  • The poison sink keeps the unparseable event with its parse error. An event the pipeline cannot read is evidence, not noise.

Why the overlay lives in the read path

  • Read-your-writes is the one piece of freshness the platform spends, so it is implemented where the cost is bounded: a short lookup of the member's own recent accepted deltas, applied to a stale projection.
  • Putting it in the write path instead would mean a synchronous projection update, which is the design this architecture exists to avoid.

Risks

  • The Bigtable row-key design is load-bearing for three different access patterns — bucket upsert, window range scan, ranked slice. Getting it wrong is a migration, not a config change.
  • Two planes mean two deploy cadences and two on-call surfaces for one product promise.