GradientBoostingClassifier<T>

Models & Types · AiDotNet.Classification.Ensemble

Gradient Boosting classifier that builds trees sequentially to correct errors.

Inheritance: ClassifierBase<T> → ProbabilisticClassifierBase<T> → EnsembleClassifierBase<T> → GradientBoostingClassifier<T>

Implements: ITreeBasedClassifier<T>

For Beginners

Gradient Boosting is one of the most powerful machine learning algorithms:

How it works:

  1. Start with an initial prediction
  2. Calculate how wrong we are
  3. Train a tree to predict our mistakes
  4. Add a fraction of this tree’s predictions
  5. Repeat, each time correcting remaining errors

Key insight: Each tree fixes what previous trees got wrong!

Tips for best results:

  • Use lower learning_rate with more n_estimators
  • Keep max_depth small (3-5) unlike Random Forest
  • Consider subsample less than 1.0 for regularization

How It Works

Gradient Boosting builds an additive model in a forward stage-wise fashion. At each stage, a regression tree is fit on the negative gradient of the loss function. For classification, this uses log loss (deviance) or exponential loss.

Example

// Create gradient boosting classifier
var options = new GradientBoostingClassifierOptions<double>();
            
// Prepare training data
var features = new Matrix<double>(new double[,] { { 1.0, 1.1 }, { 1.2, 0.9 }, { 0.8, 1.0 }, { 5.0, 5.1 }, { 5.2, 4.9 }, { 4.8, 5.0 } });
var labels = new Vector<double>(new double[] { 0, 0, 0, 1, 1, 1 });
            
// Train by sequentially fitting trees to negative gradients of loss
var result = new AiModelBuilder<double, Matrix<double>, Vector<double>>()
    .ConfigureModel(new GradientBoostingClassifier<double>(options))
    .Build(features, labels);
            
// Predict class for new sample
var newSample = new Matrix<double>(new double[,] { { 1.1, 1.0 } });
var prediction = result.Predict(newSample);
// Result is available in the returned value

Constructors

Constructor Summary
GradientBoostingClassifier(GradientBoostingClassifierOptions<T> options, IRegularization<T, Matrix<T>, Vector<T>> regularization) Initializes a new instance of the GradientBoostingClassifier class.

Properties

Property Summary
int LeafCount { get; }
int MaxDepth { get; }
int NodeCount { get; }

Methods

Method Summary
Matrix<T> PredictProbabilities(Matrix<T> input)
ModelMetadata<T> GetModelMetadata()
Vector<T> Predict(Matrix<T> input)
void Train(Matrix<T> x, Vector<T> y) Returns the model type identifier for this classifier.