OnlinePassiveAggressiveClassifier<T>

Models & Types · AiDotNet.OnlineLearning

Online Passive-Aggressive classifier for margin-based incremental learning.

Inheritance: OnlineLearningModelBase<T> → OnlinePassiveAggressiveClassifier<T>

For Beginners

PA classifiers are like strict teachers:

  • Passive: When the prediction is correct with good margin → do nothing
  • Aggressive: When wrong or uncertain → update strongly to fix the mistake

How it works:

  1. Compute margin: y × (w·x) - how confident and correct is the prediction?
  2. If margin >= 1: Correct with good margin → stay passive
  3. If margin < 1: Wrong or uncertain → aggressively update

The update is designed to:

  • Correct the mistake with minimum change to weights
  • Maintain a margin of at least 1 after update

PA variants:

  • PA: Original, no regularization (can diverge with noise)
  • PA-I: Adds slack variable, bounds the update size
  • PA-II: Adds squared penalty, smoother updates

Advantages over SGD:

  • No learning rate to tune (automatically determined)
  • Fast convergence on linearly separable data
  • Naturally handles the margin

Usage:

References:

  • Crammer et al. (2006). “Online Passive-Aggressive Algorithms”

How It Works

Passive-Aggressive (PA) algorithms are a family of online learning algorithms that aggressively update when a mistake is made but remain passive when the prediction is correct.

Constructors

Constructor Summary
OnlinePassiveAggressiveClassifier(double c, PAType type, bool fitIntercept) Gets the model type.

Methods

Method Summary
Dictionary<string, T> GetFeatureImportance() Gets the feature importance scores (absolute weights).
IFullModel<T, Matrix<T>, Vector<T>> WithParameters(Vector<T> parameters) Creates a new instance with specified parameters.
T DecisionFunction(Vector<T> x) Gets the decision function value (raw score before sign).
T GetBias() Gets the bias (intercept) term.
T GetHingeLoss(Matrix<T> x, Vector<T> y) Computes the hinge loss on the provided data.
T PredictBinary(Vector<T> x) Converts predictions to 0/1 format for compatibility.
T PredictSingle(Vector<T> x) Predicts the class label for a single sample.
Vector<T> GetWeights() Gets the weights vector.
void PartialFit(Vector<T> x, T y) Updates the model with a single training example.
void Reset() Resets the model to its initial state.