SequentialLRScheduler
Models & Types ยท AiDotNet.LearningRateSchedulers
Chains multiple learning rate schedulers together in sequence.
Inheritance: LearningRateSchedulerBase โ SequentialLRScheduler
For Beginners
Sometimes you want different learning rate strategies at different points in training. For example, you might want linear warmup for the first 1000 steps, then cosine annealing for the next 9000 steps. This scheduler lets you chain multiple schedulers together, specifying when to switch from one to the next.
How It Works
SequentialLR allows you to compose multiple schedulers, each running for a specified number of steps. This is useful for complex training schedules that combine different strategies at different phases of training.
Example
using AiDotNet.LearningRateSchedulers;
// Warmup for 1000 steps, then cosine annealing for 9000 steps
var schedulers = new List<ILearningRateScheduler>
{
new LinearWarmupScheduler(0.001, 1000, 1000),
new CosineAnnealingLRScheduler(0.001, 9000)
};
var milestones = new[] { 1000 }; // The cosine starts, at its own step 0, once 1000 steps have run
var scheduler = new SequentialLRScheduler(schedulers, milestones);
Constructors
| Constructor | Summary |
|---|---|
SequentialLRScheduler(IList<ILearningRateScheduler> schedulers, int[] milestones) |
Initializes a new instance of the SequentialLRScheduler class. |
Properties
| Property | Summary |
|---|---|
ILearningRateScheduler CurrentScheduler { get; } |
Gets the current active scheduler. |
int CurrentSchedulerIndex { get; } |
Gets the current active scheduler index. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, object> GetState() |
|
double Step() |
|
void LoadState(Dictionary<string, object> state) |
|
void Reset() |