GradientBoostingClassifier<T>
Models & Types · AiDotNet.Classification.Ensemble
Gradient Boosting classifier that builds trees sequentially to correct errors.
Inheritance: ClassifierBase<T> → ProbabilisticClassifierBase<T> → EnsembleClassifierBase<T> → GradientBoostingClassifier<T>
Implements: ITreeBasedClassifier<T>
For Beginners
Gradient Boosting is one of the most powerful machine learning algorithms:
How it works:
- Start with an initial prediction
- Calculate how wrong we are
- Train a tree to predict our mistakes
- Add a fraction of this tree’s predictions
- Repeat, each time correcting remaining errors
Key insight: Each tree fixes what previous trees got wrong!
Tips for best results:
- Use lower learning_rate with more n_estimators
- Keep max_depth small (3-5) unlike Random Forest
- Consider subsample less than 1.0 for regularization
How It Works
Gradient Boosting builds an additive model in a forward stage-wise fashion. At each stage, a regression tree is fit on the negative gradient of the loss function. For classification, this uses log loss (deviance) or exponential loss.
Example
// Create gradient boosting classifier
var options = new GradientBoostingClassifierOptions<double>();
// Prepare training data
var features = new Matrix<double>(new double[,] { { 1.0, 1.1 }, { 1.2, 0.9 }, { 0.8, 1.0 }, { 5.0, 5.1 }, { 5.2, 4.9 }, { 4.8, 5.0 } });
var labels = new Vector<double>(new double[] { 0, 0, 0, 1, 1, 1 });
// Train by sequentially fitting trees to negative gradients of loss
var result = new AiModelBuilder<double, Matrix<double>, Vector<double>>()
.ConfigureModel(new GradientBoostingClassifier<double>(options))
.Build(features, labels);
// Predict class for new sample
var newSample = new Matrix<double>(new double[,] { { 1.1, 1.0 } });
var prediction = result.Predict(newSample);
// Result is available in the returned value
Constructors
| Constructor | Summary |
|---|---|
GradientBoostingClassifier(GradientBoostingClassifierOptions<T> options, IRegularization<T, Matrix<T>, Vector<T>> regularization) |
Initializes a new instance of the GradientBoostingClassifier class. |
Properties
| Property | Summary |
|---|---|
int LeafCount { get; } |
|
int MaxDepth { get; } |
|
int NodeCount { get; } |
Methods
| Method | Summary |
|---|---|
Matrix<T> PredictProbabilities(Matrix<T> input) |
|
ModelMetadata<T> GetModelMetadata() |
|
Vector<T> Predict(Matrix<T> input) |
|
void Train(Matrix<T> x, Vector<T> y) |
Returns the model type identifier for this classifier. |