OneCycleLRScheduler
Models & Types · AiDotNet.LearningRateSchedulers
Implements the 1cycle learning rate policy.
Inheritance: LearningRateSchedulerBase → OneCycleLRScheduler
For Beginners
The 1cycle policy is like warming up before a workout, going full intensity during the workout, and then cooling down. The learning rate starts low (warmup), ramps up to a maximum (peak training), and then decreases to very low values (fine-tuning). This approach often allows training with higher maximum learning rates and can achieve better results in fewer epochs.
How It Works
The 1cycle policy starts with a low learning rate, increases it to a maximum, then decreases it again. This approach has been shown to enable faster training and better final performance, especially when combined with momentum cycling.
Based on the paper “Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates” by Leslie N. Smith and Nicholay Topin.
Example
using AiDotNet.LearningRateSchedulers;
// 1cycle policy over 100 epochs with peak LR of 0.1
var scheduler = new OneCycleLRScheduler(
maxLearningRate: 0.1,
totalSteps: 100,
pctStart: 0.3, // 30% warmup
divFactor: 25, // Start LR = 0.1/25 = 0.004
finalDivFactor: 1e4 // End LR = 0.1/10000 = 0.00001
);
Constructors
| Constructor | Summary |
|---|---|
OneCycleLRScheduler(double maxLearningRate, int totalSteps, double pctStart, double divFactor, double finalDivFactor, AnnealingStrategy annealStrategy) |
Initializes a new instance of the OneCycleLRScheduler class. |
Properties
| Property | Summary |
|---|---|
double MaxLearningRate { get; } |
Gets the maximum learning rate. |
double PctStart { get; } |
Gets the percentage of steps for warmup. |
int TotalSteps { get; } |
Gets the total number of steps. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, object> GetState() |