Trees & Ensembles
Impurity splitting, bagging, random forests, gradient boosting, and the engineering inside XGBoost and LightGBM.
4concepts
62flashcards
30minutes of reading
- 01 Bagging and Random Forests Why averaging unstable models reduces variance, why bootstrap sampling alone is not enough, and what the extra feature subsampling in a random forest is actually buying.
- 02 Why Trees Still Beat Deep Nets on Tabular Data The three inductive biases that separate tree ensembles from neural networks on tabular problems, and the specific dataset conditions under which the ordering has been observed to flip.