Leaderboard & Counting Service  ·  View 11 of 21  ·  Data

Storage Zones by Ownership

Three zones with three different recovery stories: backed up, rebuilt, or simply lost.

Editable source SVG draw.io All views
System of record — nothing else is Live log Event log Pub/Sub, 31 d, RPO 5 s Replayable history Event archive, hot GCS standard, 90 d Event archive, cold GCS archive, 13 mo Projections — droppable, rebuildable from the log Aggregates Bucket store Bigtable, 400 d Shard map per hot key Ranked views Ranked store Bigtable, versioned Rank histogram value buckets Serving cache Rank cache Memorystore, no RPO Exact and transactional — never shares a store with a counter Configuration Counter definitions Spanner, versioned Tenancy & quotas Spanner Results and evidence Closed standings immutable, 5 y, RPO 0 Retraction records append-only, 5 y Admin audit log 5 y Analytics warehouse BigQuery, derived Member directory identity, not ours continuous replay rebuild build vN warm on publish export Leaderboard & Counting Service — Storage Zones by Ownership Queue / topic Data store External / third party event / async batch synchronous Three zones, three different recovery stories: the log is backed up, the projections are rebuilt, and the cache is simply lost. A store that cannot be put in one of those three is in the wrong zone. v 1.0 · owner Platform Architecture · date 2026-10

Decisions

  • The high-volume counters and the exact transactional state never share a store. One needs cheap splittable rows at 3.5 billion keys; the other needs strong consistency at a few million rows, and a store that is good at both is good at neither at this ratio.
  • The cache carries no RPO. Losing it entirely is a latency event by definition, which is what makes it safe to treat as capacity rather than as state.
  • Closed standings are a separate store from ranked views, because the thing a reward is paid against must not be reachable by a projection build.

Retention (assumptions)

  • Log 90 days hot plus 13 months archive; bucket aggregates 400 days; closed standings and audit 5 years.
  • RPO 5 s for accepted events, RPO 0 for standings and configuration; RTO 10 min read path, 30 min full rebuild.

Risks

  • A rebuild reads the archive at ≥ 10× real time, which is a large, bursty read against the same storage the ingest path is writing.
  • 400 days of hourly buckets for 3.5 billion counters is the dominant storage line; the tiering rule is what keeps it affordable and it is easy to get wrong.
◀  Counting Data Flow All views Data Model  ▶
Solution Architecture v1.0 · Google Cloud · Platform Architecture · 2026-1011-storage-zones