StepLRScheduler
Models & Types ยท AiDotNet.LearningRateSchedulers
Decays the learning rate by a factor (gamma) every specified number of steps.
Inheritance: LearningRateSchedulerBase โ StepLRScheduler
For Beginners
This scheduler reduces the learning rate by a fixed amount at regular intervals. For example, you might reduce the learning rate by 10x every 30 epochs. This is like slowing down periodically as you get closer to your destination, making your adjustments more precise as training progresses.
How It Works
StepLR is one of the simplest and most commonly used learning rate schedulers. It multiplies the learning rate by gamma every step_size epochs/steps.
Formula: lr = base_lr * gamma^(floor(step / step_size))
Example
using AiDotNet.LearningRateSchedulers;
// Reduce LR by 10x every 30 epochs
var scheduler = new StepLRScheduler(
baseLearningRate: 0.1,
stepSize: 30,
gamma: 0.1
);
for (int epoch = 0; epoch < 100; epoch++)
{
double lr = scheduler.CurrentLearningRate; // hand this to your optimizer for the epoch
scheduler.Step(); // advance the schedule once the epoch is done
}
Constructors
| Constructor | Summary |
|---|---|
StepLRScheduler(double baseLearningRate, int stepSize, double gamma, double minLearningRate) |
Initializes a new instance of the StepLRScheduler class. |
Properties
| Property | Summary |
|---|---|
double Gamma { get; } |
Gets the multiplicative factor of learning rate decay. |
int StepSize { get; } |
Gets the step size (period of learning rate decay). |
Methods
| Method | Summary |
|---|---|
Dictionary<string, object> GetState() |