Data Modelling & Storage
Columnar formats, table formats and the lakehouse, partitioning, and modelling choices that decide query cost.
5concepts
60flashcards
36minutes of reading
- 01 Partitioning, Clustering and File Sizing The three physical layout decisions that determine query cost, why partitioning on a high-cardinality column is the classic disaster, and how to reason about the right file size.
- 02 Predicate Pushdown and Column Statistics How a query engine avoids reading data it can prove is irrelevant, the hierarchy of pruning from partition to page, and why statistics that exist can still be useless.