Encyclopedia · 176 concepts

Math & Optimization · intermediate · concept 45 of 176

Boosting (XGBoost, LightGBM)

An ensemble technique that combines many weak learners (usually decision trees) sequentially, with each new tree correcting the errors of previous ones. XGBoost and LightGBM dominate Kaggle competitions.

Key terms

Gradient boostingXGBoostLightGBMCatBoostEnsemble

Learn these first

Where you meet it in the real world

Tabular data competitions, ranking systems, click prediction

Videos

AdaBoost, Clearly Explained

StatQuest with Josh Starmer · YouTube

17. Learning: Boosting

MIT OpenCourseWare · YouTube

Courses, papers, and more