CyclicLRScheduler
Models & Types · AiDotNet.LearningRateSchedulers
Implements cyclical learning rate policy.
Inheritance: LearningRateSchedulerBase → CyclicLRScheduler
For Beginners
Instead of always decreasing the learning rate, cyclic learning rates go up and down in cycles. The idea is that periodically increasing the learning rate can help the model escape local minima (suboptimal solutions) and explore better solutions. Think of it like occasionally taking bigger jumps while hiking to avoid getting stuck in small valleys.
How It Works
CyclicLR cycles the learning rate between two boundaries with a constant frequency. This approach can help escape local minima and find better solutions by periodically increasing the learning rate.
Based on the paper “Cyclical Learning Rates for Training Neural Networks” by Leslie N. Smith.
Example
using AiDotNet.LearningRateSchedulers;
// Triangular mode cycling between 0.001 and 0.1
var scheduler = new CyclicLRScheduler(
baseLearningRate: 0.001,
maxLearningRate: 0.1,
stepSizeUp: 2000,
mode: CyclicLRScheduler.CyclicMode.Triangular
);
Constructors
| Constructor | Summary |
|---|---|
CyclicLRScheduler(double baseLearningRate, double maxLearningRate, int stepSizeUp, int? stepSizeDown, CyclicMode mode, double gamma) |
Initializes a new instance of the CyclicLRScheduler class. |
Properties
| Property | Summary |
|---|---|
CyclicMode Mode { get; } |
Cyclic mode (Triangular / Triangular2 / ExponentialRange). |
double MaxLearningRate { get; } |
Gets the maximum learning rate. |
int CycleCount { get; } |
Gets the current cycle count. |
int StepSizeDown { get; } |
Gets the step size for decreasing phase. |
int StepSizeUp { get; } |
Gets the step size for increasing phase. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, object> GetState() |