An order-lines fact table holds 40 million rows per day for three years across 18 columns and is partitioned by order date. In Parquet it averages roughly 36 bytes per row compressed and columns are of similar width. A dashboard query reads three columns over the last 90 days. Roughly how many bytes does it scan?
Show the full answer Hide the answer
The arithmetic
State the assumptions first, because the answer is only as good as they are.
- Rows actually read: partitioning by order date means the engine reads 90 days, not 1,095. 40M × 90 = 3.6 billion rows.
- Bytes per row actually read: 36 bytes covers all 18 columns. Three columns of similar width are roughly 3/18 of that, so about 6 bytes per row.
- Multiply: 3.6e9 × 6 ≈ 21.6 GB, call it 20–25 GB.
Two effects push the real number around, and both are worth knowing: a column of high-cardinality strings can be several times the average width, and row-group statistics can prune further if the data happens to be sorted on a predicate column.
When this estimate is the wrong tool
An estimate decides whether a design is plausible, not whether a query is correctly written. If the number lands within a factor of two of a budget ceiling, stop estimating and measure: every engine reports bytes scanned for a completed query, and that measurement takes a minute.
Which assumption dominates the error
The per-column width. Row count and day count are known facts; the 1/18 share is the guess. If one of the three columns is a UUID stored as text it alone can be 20–30 bytes compressed, tripling the answer. Before defending an estimate like this, check the widest column, not the row count.
What the number rules in or out
At roughly 20 GB per run, a consumption-priced engine — on-demand scan pricing has sat in the order of $5 per TB since the mid-2010s — costs on the order of 10 cents a query. Hourly refresh is fine. The same query without partition pruning scans 1,095 days: around 260 GB, twelve times the cost, and the dashboard now shows up in the monthly bill. That gap, not the absolute number, is the design argument for keeping the partition predicate in the query and out of a wrapping function.
Why the other options fail
- About 0.6 GB is one day rather than 90. It is the mistake of estimating the pipeline's increment instead of the query's range, and it is how dashboards get approved and then surprise everyone.
- About 130 GB counts all 18 columns for 90 days. That is row-store thinking: correct for a heap scan in an operational database, wrong for a columnar format where the engine opens only the column chunks it needs.
- About 1.4 TB is the full three years across all columns: no partition pruning and no column
pruning. This is what you actually get when someone writes
WHERE date_trunc('month', order_date) = ..., because wrapping the partition column in a function defeats pruning.