intermediate
2 min answer
An analytics platform separates compute from storage. What does that actually buy architecturally, and what problems does it not solve?
Show the full answer Hide the answer
What it buys
- Independent scaling of two resources that grow at different rates. Data volume grows monotonically; query demand is spiky and diurnal. Coupling them means paying for compute to hold data, which is the central inefficiency of the shared-nothing warehouse model.
- Workload isolation without data duplication. Several compute clusters read the same storage, so the finance team's month-end run cannot slow the product team's dashboards, and neither needs its own copy of the data.
- Elasticity. Compute can be resized or suspended in seconds, so cost tracks usage rather than peak.
- Clean cost attribution. Each workload's compute is separately measurable, which is what actually changes team behaviour.
What it does not solve
- Query cost. A badly written query scanning a billion rows costs the same whatever the architecture. The separation makes it easier to see who spent the money, which is helpful, and does not reduce it.
- Data layout. Partitioning, clustering and file sizes still determine how much data a query must read, and they are the dominant factor in both latency and cost. Separation does not excuse you from modelling.
- Concurrency within a cluster. Many simultaneous queries on one warehouse still queue. Isolation is between clusters, not within one.
- The metadata layer, which is shared and can become a bottleneck at extreme object counts.
- Governance. Shared storage with many compute clusters makes access control more important, not less, because the perimeter is now a policy rather than a deployment.
The economic trap
Elasticity makes it trivially easy to spend money. A warehouse that autoscales to satisfy a runaway dashboard refreshing every thirty seconds will do exactly that, silently. The architectural response is per-workload budgets, query timeouts, result caching and auto-suspend — and these are the controls teams add after the first surprising invoice rather than before.
Separation of compute and storage moves the constraint from capacity to cost, and cost is a softer, easier-to-ignore constraint than a full disk. That is its main hidden hazard.