concept

Partition Pruning

The query engine skipping entire partitions whose values cannot match the filter, which is where most large-table query performance comes from.

performancepartitioningquery

Pruning is the reason partitioning exists, and understanding when it fails is more valuable than knowing the syntax. The engine can skip a partition only when it can prove from metadata that no row inside could satisfy the predicate — which requires the filter to be on the partition column, in a form the planner recognises.

The common ways it silently stops working: applying a function to the partition column in the predicate, so the engine cannot map the filter onto partition values; joining on the partition column instead of filtering on it; a data type mismatch forcing a cast; and querying a partitioned table through a view that obscures the column.

The result is a full scan with the same query returning the same answer, so nothing looks broken — the bill and the runtime are the only symptoms. Checking the query plan for the number of partitions scanned is the diagnostic, and it belongs in review for any query over a large table.

Choosing the partition column is the design decision underneath: it should match how the data is actually filtered, usually a date, with cardinality low enough to avoid millions of tiny partitions and high enough to eliminate most of the table. Partitioning by a high-cardinality identifier is the classic error, and it produces the small file problem as a bonus.