Math & Optimization · intermediate · concept 45 of 176
Boosting (XGBoost, LightGBM)
An ensemble technique that combines many weak learners (usually decision trees) sequentially, with each new tree correcting the errors of previous ones. XGBoost and LightGBM dominate Kaggle competitions.
Key terms
Gradient boostingXGBoostLightGBMCatBoostEnsemble
Learn these first
Where you meet it in the real world
Tabular data competitions, ranking systems, click prediction
Videos
▶ AdaBoost, Clearly Explained ↗
StatQuest with Josh Starmer · YouTube
▶ 17. Learning: Boosting ↗
MIT OpenCourseWare · YouTube
Guides and articles
Courses, papers, and more