OnlinePassiveAggressiveRegressor<T>

Models & Types · AiDotNet.OnlineLearning

Online Passive-Aggressive regressor for epsilon-insensitive incremental regression.

Inheritance: OnlineLearningModelBase<T> → OnlinePassiveAggressiveRegressor<T>

For Beginners

PA regressors work like strict teachers for continuous predictions:

  • Passive: When the prediction error is within ε → do nothing
  • Aggressive: When error exceeds ε → update strongly to reduce the error

How it works:

  1. Compute prediction: ŷ = w·x + b
  2. Compute error: e = |y - ŷ|
  3. If e ≤ ε (epsilon): Acceptable error → stay passive
  4. If e > ε: Too much error → aggressively update weights

The ε (epsilon) parameter defines an “acceptable error zone” similar to epsilon-insensitive loss in SVR.

PA variants:

  • PA: Original, no regularization (can diverge with noise)
  • PA-I: Adds slack variable, bounds the update size
  • PA-II: Adds squared penalty, smoother updates

Advantages over standard online regression:

  • No learning rate to tune (automatically determined)
  • Robust to small errors (epsilon-insensitive)
  • Fast convergence on well-behaved data

Usage:

References:

  • Crammer et al. (2006). “Online Passive-Aggressive Algorithms”

How It Works

Passive-Aggressive (PA) regression algorithms are a family of online learning algorithms that aggressively update when the prediction error exceeds a threshold but remain passive when the error is within tolerance.

Constructors

Constructor Summary
OnlinePassiveAggressiveRegressor(double c, double epsilon, PAType type, bool fitIntercept, int batchEpochs) Gets the model type.

Properties

Property Summary
double Epsilon { get; } Gets the epsilon parameter (insensitivity zone).
int BatchEpochs { get; } Gets the number of passes used by the offline Train wrapper.

Methods

Method Summary
Dictionary<string, T> GetFeatureImportance() Gets the feature importance scores (absolute weights).
IFullModel<T, Matrix<T>, Vector<T>> WithParameters(Vector<T> parameters) Creates a new instance with specified parameters.
T GetBias() Gets the bias (intercept) term.
T GetEpsilonInsensitiveLoss(Matrix<T> x, Vector<T> y) Computes the epsilon-insensitive loss on the provided data.
T GetMeanSquaredError(Matrix<T> x, Vector<T> y) Computes the mean squared error on the provided data.
T PredictSingle(Vector<T> x) Predicts the value for a single sample.
Vector<T> GetWeights() Gets the weights vector.
void PartialFit(Matrix<T> x, Vector<T> y)
void PartialFit(Vector<T> x, T y)
void Reset() Resets the model to its initial state.
void Train(Matrix<T> x, Vector<T> y)