Reasoning & Evaluation
24 min
Inside Gradient-Boosted Trees: The Engineering That Made XGBoost, LightGBM and CatBoost Win Tabular ML
XGBoost, LightGBM and CatBoost minimise the same objective with the same kind of tree. What separates them is bookkeeping: XGBoost turned two sums of derivatives into a split score, LightGBM made those sums cheap, and CatBoost ma…