Tabular Deep Learning
Why trees still win, attention over columns, prior-fitted networks and the benchmarks that decide the argument.
6concepts
67flashcards
44minutes of reading
- 01 Attention Over Columns How transformer architectures are adapted to tables, what per-feature embeddings buy, and why the attention mechanism addresses the feature selection problem specifically.
- 02 Benchmarking Tabular Methods Honestly Why tabular comparisons disagree so persistently, the tuning asymmetry that produces most of the disagreement, and what a comparison needs to support its conclusion.
- 03 Deep Learning for High-Cardinality Categoricals The one tabular regime where networks clearly win, why embeddings handle millions of levels that trees cannot, and the engineering that dominates such systems.
- 04 Numerical Feature Embeddings for Tabular Models Why feeding a raw scalar into an MLP handicaps it on tabular data, how piecewise linear encodings and trainable periodic features turn each number into a vector, and what Gorishniy et al. (2022) measured when they did.
- 05 Prior-Fitted Networks and In-Context Tabular Learning How a model trained entirely on synthetic datasets can classify a real one without fitting, what approximating the Bayesian posterior in a forward pass means, and the constraints that follow.
- 06 The Inductive Bias Mismatch on Tabular Data The three properties of tabular data that neural networks handle badly and trees handle naturally, why this is a bias mismatch rather than a capacity problem, and what it implies for architecture design.