SequentialLRScheduler

Models & Types ยท AiDotNet.LearningRateSchedulers

Chains multiple learning rate schedulers together in sequence.

Inheritance: LearningRateSchedulerBase โ†’ SequentialLRScheduler

For Beginners

Sometimes you want different learning rate strategies at different points in training. For example, you might want linear warmup for the first 1000 steps, then cosine annealing for the next 9000 steps. This scheduler lets you chain multiple schedulers together, specifying when to switch from one to the next.

How It Works

SequentialLR allows you to compose multiple schedulers, each running for a specified number of steps. This is useful for complex training schedules that combine different strategies at different phases of training.

Example

using AiDotNet.LearningRateSchedulers;
// Warmup for 1000 steps, then cosine annealing for 9000 steps
var schedulers = new List<ILearningRateScheduler>
{
    new LinearWarmupScheduler(0.001, 1000, 1000),
    new CosineAnnealingLRScheduler(0.001, 9000)
};
var milestones = new[] { 1000 };  // The cosine starts, at its own step 0, once 1000 steps have run
var scheduler = new SequentialLRScheduler(schedulers, milestones);

Constructors

Constructor Summary
SequentialLRScheduler(IList<ILearningRateScheduler> schedulers, int[] milestones) Initializes a new instance of the SequentialLRScheduler class.

Properties

Property Summary
ILearningRateScheduler CurrentScheduler { get; } Gets the current active scheduler.
int CurrentSchedulerIndex { get; } Gets the current active scheduler index.

Methods

Method Summary
Dictionary<string, object> GetState()
double Step()
void LoadState(Dictionary<string, object> state)
void Reset()