Every morning at 09:00 executive dashboards are slow. Investigation shows a data scientist's exploratory query. How do you fix this permanently?
Show the full answer Hide the answer
What the interviewer is testing
Whether you fix the contention structurally rather than socially.
Why the social fix fails
Asking the data scientist to avoid mornings works until the next data scientist. On a shared cluster this is a queueing problem with no clean answer, because the offending query is already running and consuming resources that the dashboards need.
The structural fix
Separate compute per workload class against the same tables — which separation of storage from compute made straightforward:
| Workload | Compute | Configuration |
|---|---|---|
| Executive dashboards | Small, always on | Strict concurrency, short timeout, result caching |
| Scheduled transformations | Elastic | Sized for throughput, off-peak where possible |
| Exploratory analysis | Elastic, aggressive auto-suspend | Byte-scan limit, hard timeout |
| Embedded / external analytics | Isolated | Separate entirely, since a customer-facing path must not share |
The trade is utilisation against predictability: several small pools idle more than one large pool, and they make cost attributable per workload — which is usually worth more than the utilisation.
The additional controls
Query guards on the exploratory pool: a byte-scan limit and a timeout, so an accidental full scan of the largest table produces an error rather than an outage and an invoice.
Result caching and materialised aggregates for the dashboards, so the morning load is largely served from cache and does not compete at all.
What a strong answer adds
Checking whether the dashboards should be hitting the warehouse at 09:00 at all. Executive dashboards querying raw tables live is a common and avoidable design: pre-computing the aggregates on a schedule makes the morning read trivially cheap and removes the contention entirely.
Common weak answers
Scaling the shared cluster, which raises the cost and leaves the contention. Restricting data scientist access, which damages the platform's purpose.