An analytics platform's cost is growing faster than its usage. What causes it, and which controls work without blocking analysts?

The usual causes

• Full scans from poor data layout. Partitioning and clustering that do not match query filters mean nothing can be pruned. The single largest cause, and the one addressed last because adding compute also works.
• Dashboards on aggressive refresh schedules, most of which nobody is looking at — a dashboard refreshing every minute for an audience that checks it weekly.
• SELECT * on wide columnar tables, which reads every column when the query needs four.
• Repeated identical queries that could be cached or materialised.
• Idle compute left running, and auto-suspension set too generously.
• Development and exploration on production-sized data, where a sample would answer the question.
• Uncontrolled data retention, with raw data kept at full fidelity forever.
• Small files defeating pruning and multiplying per-file overhead, from uncompacted streaming ingestion.

Controls that work

• Fix the layout first. Partition and cluster on the columns queries actually filter by, derived from query logs rather than from assumptions. This frequently reduces cost by an order of magnitude and improves performance simultaneously, which is the rare intervention with no trade-off.
• Materialise the expensive repeated queries as tables, so the cost is paid once rather than per viewer.
• Auto-suspend aggressively, accepting the first-query latency — an explicit trade, and one that should be measured rather than guessed.
• Query timeouts and scan limits per workload class, which stop a runaway query without blocking legitimate work.
• Right-sized clusters per workload, since an oversized cluster does not make a badly-laid-out query cheap.
• Tiered retention, with raw data aged into cheaper storage and aggregates retained longer.
• Compaction scheduled and monitored, since its absence degrades everything gradually and silently.
• Sampled datasets for exploration, which analysts adopt willingly when they are faster.

Controls that block analysts and should be avoided

• Approval before running a query, which destroys the exploratory work the platform exists to enable.
• Hard per-team budgets that stop work mid-month, which produce workarounds and resentment.
• Removing self-service in favour of a central team, which recreates the bottleneck the platform replaced.

What makes cost control stick

Attribution and visibility, in the analysts' own tools. A query interface showing estimated cost before execution and actual cost afterwards changes behaviour immediately and without any policy — people optimise what they can see, and analysts are as responsive to this as engineers.

Per-team dashboards with unit metrics, not just totals: cost per active user, per dashboard, per pipeline run. A team whose cost rose 40% while its usage rose 80% is improving, and a total-only view punishes them for growth.

And the framing for the business conversation: not "why do we spend this?" but "what analysis could we not do if we spent less, and what is that worth?" — which converts a cost argument into a value argument, where it belongs.