An analytics platform ingests billions of events while customers run arbitrary segmentation queries. How should the two workloads be isolated, and when should pre-aggregation be introduced?

Why isolation is the first decision

Ingestion is a continuous high-volume write workload that must never fall behind. Query is spiky, arbitrary, and can consume unbounded resources. Sharing infrastructure between them means one customer's expensive query stalls ingestion for everyone, and ingestion falling behind is worse than a slow query because the data gap is permanent and visible.

The separation:

• Ingestion writes to storage; queries read from it. Different compute, different scaling signals, no shared pool.
• Ingestion buffered through a durable queue, so a storage slowdown does not become data loss and the spikiness is absorbed before it reaches the store.
• Query concurrency limited per customer, with queueing and a visible position rather than unbounded parallelism.
• A separate pool for very expensive queries, so an unbounded scan does not sit alongside interactive dashboard refreshes.

When pre-aggregation earns its cost

Pre-aggregation is not free: it fixes the questions in advance, requires backfilling when definitions change, and produces a maintenance burden proportional to the number of aggregates.

It earns its cost when: the same query is run repeatedly (a dashboard refreshing every minute for a hundred users), the raw scan is genuinely expensive, and the aggregate's definition is stable.

It does not earn its cost for exploratory analysis, which is by definition unpredictable — and building aggregates for exploration produces a large set that answers questions nobody asks while the actual question still requires a raw scan.

The design that resolves it

A columnar store with good compression and partitioning for raw events, plus a small set of materialised aggregates for the known-hot queries. The aggregates cover the dashboards; the raw data covers exploration.

The distinguishing signal is repetition, not cost. An expensive query run once a month should stay a raw scan; a cheap query run ten thousand times a day is a candidate for pre-aggregation.

The cost control that changes behaviour

Per-customer query cost attribution and quotas. Analytics workloads have no natural ceiling — customers will run whatever the platform permits — and technical limits produce complaints while visible cost produces changed query patterns. This is the lever that works, and it is a product decision rather than an engineering one.