AdaBoost Explained From Its Original Paper

Author(s): Christian Guerra

Originally published on Towards AI.

This publication is meant to show a very popular ML algorithm in complete detail, how it works, the math behind it, how to execute it in Python and an explanation of the proofs of the original paper. There will be math and code, but it is written in a way that allows you to decide which are the fun parts.

A bit on the origins of the algorithm: It was proposed by Yoav Freund and Robert E. Schapire in a 1997 paper, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting”—a beautiful and brilliant publication for an effective and useful algorithm.

Let’s start with the pros, cons, and uses of AdaBoost.

Advantages: improves performance and achieves higher accuracy than a single model. It reduces overfitting compared to some other machine learning algorithms.

Disadvantages: AdaBoost can be sensitive to noisy data and outliers. It requires careful tuning, and the performance can depend on the choice of weak learners and the number of iterations. It cannot be parallelized (or only partially), since each predictor can only be trained after the previous predictor has been trained and evaluated. As a result, it does not scale as well as bagging or pasting.

Applications: image recognition, text classification,… Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI