AdaBoost Multiclass Classifier¶
AdaBoost (short for “Adaptive Boosting”) is a popular boosting classification algorithm. The AdaBoost algorithm performs well on a variety of data sets except some noisy data ([Friedman98], [Zhu2005]). The library supports two methods for the algorithms:
SAMME, or Stagewise Additive Modeling using a Multi-class Exponential loss function [Zhu2005]
SAMME.R, which is a modification of SAMME method for Real-valued returned probabilities from weak learner
Details¶
Given
Training Stage¶
The following scheme shows the major steps of the SAMME algorithm:
Initialize weights
forFor
:Train the weak learner
using weights .Choose a confidence value
, whereUpdate
, where is a normalization factor.
Output the final hypothesis:
Note
SAMME algorithm in case of binary classification is equal to the AdaBoost algorithm from [Friedman98].
The following scheme shows the major steps of the SAMME.R algorithm:
Initialize weights
forFor
:Train the weak learner
using weights .Receive the weighed class probability estimates from weak learner:
For
, set :For
, update :where
is a normalization factor, ,
Output the final hypothesis:
Prediction Stage¶
Given the AdaBoost classifier and
Given the AdaBoost classifier and
where
Batch Processing¶
AdaBoost classifier follows the general workflow described in Classification Usage Model.
Training¶
For a description of the input and output, refer to Classification Usage Model. At the training stage, an AdaBoost classifier has the following parameters:
Parameter |
Default Value |
Description |
---|---|---|
|
|
The floating-point type that the algorithm uses for intermediate computations. Can be |
|
|
Available methods for computation of the AdaBoost algorithm:
|
|
Pointer to an object of the classification stump training class |
Pointer to the training algorithm of the weak learner. By default, a classification stump weak learner is used. |
|
Pointer to an object of the classification stump prediction class |
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used. |
|
AdaBoost training accuracy. |
|
|
The maximal number of iterations for the algorithm. |
|
|
Multiplier for each classifier to shrink its contribution. |
|
|
The number of classes. |
|
|
The 64-bit integer flag that specifies which extra characteristics of AdaBoost to compute.
Current version of the library only provides the following option: |
Output¶
In addition to classifier output, AdaBoost calculates the result described below.
Pass the Result ID
as a parameter to the methods that access the result of your algorithm.
For more details, see Algorithms.
Result ID |
Result |
---|---|
|
A numeric table Note By default, this result is an object of the |
Prediction¶
For a description of the input and output, refer to Classification Usage Model. At the prediction stage, an AdaBoost classifier has the following parameters:
Parameter |
Default Value |
Description |
---|---|---|
|
|
The floating-point type that the algorithm uses for intermediate computations. Can be |
|
|
Performance-oriented computation method, the only method supported by the AdaBoost classifier at the prediction stage. |
|
Pointer to an object of the classification stump prediction class |
Pointer to the prediction algorithm of the weak learner. By default, a classification stump weak learner is used. |
|
The number of classes. |