Leaderboard & Counting Service · View 10 of 21 · Data
Decisions
- The log is written twice by design: a 31-day live topic for streaming delivery and an archive for 90-day replay. One is a transport, the other is the record; conflating them means the retention of the record is set by the broker's limits.
- Uniqueness is counted with a sketch rather than a member set, because storing the member set is what makes a distinct-viewer counter cost more than the view counter it accompanies.
- Late events beyond the declared horizon land in a visible late bucket rather than being dropped or silently folded in.
Numbers (assumptions)
- Event archive 25 TB per 90 days after compression; hot for 90 days, archive class to 13 months.
- Approximate unique counters ≤ 2% relative error at p95, ≤ 5% at p99.
- Finest bucket is one hour; a 7-day window composes from 168 of them.
Risks
- A bucket-addition read for a 30-day window is a 720-bucket fan-in. Either coarser roll-ups are maintained or the window set stays small.
- Writing the log twice doubles the ingest cost floor and adds a consistency question between the two copies at the boundary of a replay.
