ReduceOnPlateauScheduler

Models & Types · AiDotNet.LearningRateSchedulers

Reduces learning rate when a metric has stopped improving.

Inheritance: LearningRateSchedulerBase → ReduceOnPlateauScheduler

For Beginners

Unlike other schedulers that follow a fixed schedule, this one watches your model’s performance and only reduces the learning rate when training gets “stuck” (plateaus). If the model keeps improving, it keeps the learning rate the same. If improvement stops for a while (patience epochs), it reduces the learning rate to allow finer adjustments. Think of it like slowing down only when you notice you’re not making progress.

How It Works

ReduceOnPlateau monitors a quantity (usually validation loss) and reduces the learning rate when no improvement is seen for a ‘patience’ number of evaluations. This is a reactive scheduler that adapts based on training progress rather than a fixed schedule.

This scheduler requires you to call the Step(metric) overload with the monitored value.

Example

using AiDotNet.LearningRateSchedulers;
var scheduler = new ReduceOnPlateauScheduler(
    baseLearningRate: 0.1,
    factor: 0.1,
    patience: 10,
    mode: ReduceOnPlateauScheduler.Mode.Min
);
            
for (int epoch = 0; epoch < 100; epoch++)
{
    double lr = scheduler.CurrentLearningRate;   // hand this to your optimizer for the epoch
    double valLoss = 1.0 / (epoch + 1);          // stand-in for your own validation metric
    scheduler.Step(valLoss);                     // scheduler decides whether to reduce LR
}

Constructors

Constructor Summary
ReduceOnPlateauScheduler(double baseLearningRate, double factor, int patience, double threshold, ThresholdMode thresholdMode, int cooldown, Mode mode, double minLearningRate) Initializes a new instance of the ReduceOnPlateauScheduler class.

Properties

Property Summary
double BestValue { get; } Gets the best metric value seen so far.
double Factor { get; } Gets the reduction factor.
int NumBadEpochs { get; } Gets the current number of bad epochs.
int Patience { get; } Gets the patience value.

Methods

Method Summary
Dictionary<string, object> GetState()
double Step()
double Step(double metric) Steps the scheduler with a metric value.
void LoadState(Dictionary<string, object> state)
void Reset()