A platform needs sub-second analytical queries over a continuously updating dataset. What kind of store, and what are the trade-offs?

The requirement's two halves

Continuous ingestion and low-latency analytical queries — which are opposing requirements. Analytical stores are optimised for scanning large amounts of columnar data and are typically poor at high-frequency small writes; operational stores are the reverse.

A real-time analytical store is one that has made a specific compromise between them, and choosing one means choosing which compromise fits.

What the compromises look like

• Batched ingestion with a short interval, giving near-real-time freshness with columnar read performance. The batch interval is the explicit freshness dial, and micro-batching gives most of the freshness with most of batch's simplicity.
• A separate write-optimised layer merged periodically into the read-optimised one, so recent data is queryable immediately and older data is efficiently stored. Queries read both and merge.
• Pre-aggregation at ingestion, which makes specific queries fast and fixes the questions in advance — excellent for known dashboards and useless for exploration.

The decision that determines the design

Whether the query patterns are known. If they are — a fixed set of dashboards and monitors — pre-aggregation is dramatically cheaper and faster. If they are not, the store must scan, and the design must be optimised for scanning.

Building pre-aggregations for exploration produces a large set answering questions nobody asks while the real question still requires a raw scan.

What must be isolated

Ingestion from query. A large query must not stall ingestion, because ingestion falling behind is worse than a slow query: the data gap is permanent and visible, and it occurs precisely during the period someone will want to analyse.

A durable queue in front of ingestion converts a storage slowdown into a delay rather than a loss.

The domain-specific caution

For a trading platform, the analytical store must not be on the order path. Its ingestion, its queries and its failures must be entirely separate from the transaction path — because the analytical workload is exactly the one that spikes when the market moves, which is when the trading path has least margin.

That separation is a hard architectural boundary rather than a tuning preference.