StepLRScheduler

Models & Types ยท AiDotNet.LearningRateSchedulers

Decays the learning rate by a factor (gamma) every specified number of steps.

Inheritance: LearningRateSchedulerBase โ†’ StepLRScheduler

For Beginners

This scheduler reduces the learning rate by a fixed amount at regular intervals. For example, you might reduce the learning rate by 10x every 30 epochs. This is like slowing down periodically as you get closer to your destination, making your adjustments more precise as training progresses.

How It Works

StepLR is one of the simplest and most commonly used learning rate schedulers. It multiplies the learning rate by gamma every step_size epochs/steps.

Formula: lr = base_lr * gamma^(floor(step / step_size))

Example

using AiDotNet.LearningRateSchedulers;
// Reduce LR by 10x every 30 epochs
var scheduler = new StepLRScheduler(
    baseLearningRate: 0.1,
    stepSize: 30,
    gamma: 0.1
);
            
for (int epoch = 0; epoch < 100; epoch++)
{
    double lr = scheduler.CurrentLearningRate;   // hand this to your optimizer for the epoch
    scheduler.Step();                            // advance the schedule once the epoch is done
}

Constructors

Constructor Summary
StepLRScheduler(double baseLearningRate, int stepSize, double gamma, double minLearningRate) Initializes a new instance of the StepLRScheduler class.

Properties

Property Summary
double Gamma { get; } Gets the multiplicative factor of learning rate decay.
int StepSize { get; } Gets the step size (period of learning rate decay).

Methods

Method Summary
Dictionary<string, object> GetState()