Query Consolidation

A popular row is read by everyone at once. The reads are identical, they return the same bytes, and a conventional proxy forwards every one of them. At 20,000 reads per second of a single row, 19,999 of them are work the storage engine performs for no new information.

Query consolidation removes that work at the point where it is cheapest to notice. When a query arrives and an identical query is already in flight, the new caller does not execute anything - it waits on the running execution and receives its result. Vitess, the MySQL proxy layer built at YouTube and now a CNCF project, implements this in its tablet server, and its documentation is precise about what identical means: byte-for-byte, including literals and bind variables.

Why it matters

The saving scales the right way. A slow query consolidates more callers than a fast one, because the in-flight window over which repeats accumulate is the query's own duration. As a hot read degrades, each execution serves more waiters, so the feedback loop damps rather than amplifies. That is the opposite of the usual behaviour under load and it is why the mechanism is valuable precisely during an incident.

It is also cheap to reason about, because nothing is stored. There is no key space, no memory budget, no eviction policy, and no invalidation story - the three hardest parts of caching are absent, and the maximum staleness is bounded by the duration of one query.

Implementation patterns

• Consolidate at the tablet or data-service layer, not the application, so every caller benefits without coordination.
• Cap the waiters. Vitess exposes --consolidator-query-waiter-cap, unlimited by default, and a newer option to reject over-cap queries with RESOURCE_EXHAUSTED rather than execute them independently. The cap is the backpressure.
• Restrict it to replicas where read-your-writes matters: --enable-consolidator-replicas exists for exactly that.
• Pair it with a concurrency limit per hot key when the goal is protecting a partition rather than saving work, which is the shape Discord described for its Rust data services in front of message storage.
• Normalise query text deliberately if you want hit rates: consolidation fires on exact repeats, so a predicate carrying a per-caller parameter never consolidates.

Industry example

Vitess was built at YouTube to put MySQL behind a proxy layer that could shard, rewrite and govern queries, and consolidation is one of the governors it kept. Its documented consistency caveat is the whole trade-off in one sentence: the second caller receives the first caller's result, so a read issued after your own write can be answered by an execution that started before it.

Discord's 2023 write-up on moving trillions of messages to ScyllaDB describes a related construction - Rust data services in front of the store whose main job is coalescing repeated reads of the same partition and capping concurrent traffic to it.

Failure scenarios

• Silent stale reads on a primary serving read-your-writes traffic. Nothing errors; a user edits a message and sees the old text.
• Waiter pile-up. A 9-second query with 8,000 waiters holds 8,000 connections and the memory behind them. With no cap, one slow query becomes an availability incident.
• Convoy effect. Everyone waiting on one execution means everyone's latency equals the slowest execution's, so p50 rises towards p99 for the consolidated query.
• False confidence. A team measures "queries per second to the database" falling after enabling it and concludes the hot-row problem is solved, while the row is still a single point of contention for writes.

Trade-offs

When not to use it

Do not use it on a primary that serves read-your-writes traffic, and do not use it for a workload of mostly distinct queries, where the hit rate approaches zero and only the machinery remains. For a single application with one hot row, a 200 ms in-process cache is simpler, needs no proxy feature, and produces the same curve. Consolidation earns its place when many independent callers issue the identical parameter-free read and tolerate staleness bounded by one query's duration.

Interview question

Q: Your proxy layer can share one in-flight read's result with every identical read behind it. Which of your endpoints would you enable it for, which would you refuse, and what limit do you set before turning it on?

What a strong answer covers: enabling it on replica-served, parameter-free hot reads (a trending item, a public profile, a config blob) and refusing it on anything that follows a write by the same user; recognising that the limit is the waiter cap and that unlimited waiters convert a slow query into a connection exhaustion event; noting that the maximum staleness is one query duration rather than a TTL; and knowing that it is not a cache, so the invalidation question does not arise.

Quick check

Quiz: Why does consolidation become more effective as the consolidated query gets slower? Because the in-flight window is the query's duration, so a longer execution accumulates more waiters to share it.

Flashcard: What does query consolidation break, and what flag limits the damage? Read-after-write consistency for a caller whose read starts after its own write; the waiter cap bounds how many callers can block on one execution.