A product-analytics platform serves many customers' data on shared infrastructure. What isolation is required at the storage and query layers?
Show the full answer Hide the answer
Storage isolation
- Tenant as a partition key, so a query for one customer reads only their partitions. This is both an isolation property and the dominant performance property — a query that must scan all tenants' data to filter one is unusable at scale.
- Enforced at the engine rather than in application code, so a missing filter returns nothing rather than everything. The one query that forgets is a cross-tenant data breach, and code review does not reliably catch it.
- Dedicated storage for the largest tenants where volume or contractual requirements justify it, with the same interfaces so a tenant can move between placements without a rewrite.
Query isolation
- Per-tenant concurrency limits with a visible queue position, rather than unbounded parallelism.
- Cost-weighted quotas rather than query counts, since counting queries treats a dashboard refresh and a two-year scan identically, which is exactly backwards.
- A separate pool for very expensive queries, so an unbounded scan does not sit beside interactive dashboards.
- Statement timeouts, bounding the damage from a runaway query.
The ingestion side
Ingestion must be isolated from query entirely — different compute, no shared pool — because ingestion falling behind is worse than a slow query: the data gap is permanent and visible, and it occurs precisely during the period the customer will later want to analyse.
A durable queue in front of ingestion absorbs spikiness and converts a storage slowdown into a delay rather than a loss.
The economics that follow
Analytical workloads have no natural ceiling — customers run whatever the platform permits — so per-customer cost attribution and quotas are not optional. Technical limits produce complaints; visible cost produces changed query patterns, which is the lever that works.
The cardinality trap specific to this domain
Customer-defined event properties are unbounded high-cardinality dimensions. A customer sending a unique identifier as a property multiplies storage and index cost, and the platform absorbs it silently until it does not.
A per-tenant cardinality budget with clear feedback at ingestion is the control, and rejecting with an explanation is far better than accepting and paying.