A product page needs the current computed price for one product id at 40,000 reads per second with p99 under 25 ms. A stream processor already recomputes each product's price every few seconds from competitor and inventory events. Where should the read path go?
Show the full answer Hide the answer
The deciding property
The answer is already computed, and the query is a point lookup of one key. Nothing about this read path needs filtering, grouping, ranking or ad-hoc predicates. When the question is "give me the value for key K", the serving store's only jobs are key access, durability and predictable p99 under concurrency. A key-value store with the processor as its sole writer does exactly that and nothing else, which is why it has the smallest failure surface of the four.
The design rule generalises: what to compute at write time versus read time is settled by whether the query's shape is known in advance. Known shape and bounded key space means precompute and serve by key. Unknown shape means keep the rows and aggregate at read time.
Why the other options fail
- Queryable state. Tempting because the value is already in the processor's memory and a second store feels redundant. It fails on availability coupling: state lives on the task that owns the key, so the read path needs a key-to-host map that changes on every rebalance, every rescale and every deploy. Serving availability becomes job availability, and scaling reads means scaling the job. It is the right answer when read volume is tiny and the state is too large to copy out.
- Append events and aggregate at read time. Correct when the page needs "price history for the last hour grouped by channel" and the grouping is chosen by the user. Here it re-derives a value the processor has already derived, at 40,000 times per second, and pays scan cost on every request to preserve flexibility nobody asked for.
- Postgres read replica. This is the right answer below roughly 5,000 reads per second, and the operational simplicity of one familiar database genuinely beats a second datastore at that scale. At 40,000 reads per second it is not throughput that bites but the tail: replica apply lag during a write burst, autovacuum, and connection handling all put mass in p99 exactly when prices are moving fastest. Teams who choose it end up putting a cache in front of it, which is the first option reached by a longer road.
What would flip the decision
| If this changes | Choose | Because |
|---|---|---|
| Reads fall to a few thousand per second | Postgres replica | One less system to run beats a marginal p99 gain |
| The page needs user-chosen filters over recent events | Real-time analytical store | The query shape is no longer known at write time |
| The value must be strongly consistent with the write that produced it | Processor-side transactional store | A key-value store fed asynchronously is read-your-writes only by luck |
| Key count grows into billions with a long tail | Key-value store plus a cold path | Precomputing every key stops being economic |
When this is the wrong call
Every value served from a precomputed store needs its computation time attached. A price row with no timestamp cannot be distinguished from a price row the processor stopped updating 40 minutes ago, and the page will serve the stale value with total confidence. If you cannot stamp and monitor freshness per key, read-through to the source is the honest design even at a worse p99.