AdaptiveFitnessScheduler

Models & Types · AiDotNet.LearningRateSchedulers

Adjusts the learning rate in response to whether the monitored metric improved: shrink while improving, grow while stalled.

Inheritance: LearningRateSchedulerBase → AdaptiveFitnessScheduler

For Beginners

The learning rate controls how big a step the model takes each update. This adjusts it automatically: smaller steps while the model is improving so it can settle precisely, larger steps when it is stuck so it can escape.

How It Works

This is the rule optimizers previously applied inline when UseAdaptiveLearningRate was set — improving fitness multiplies the rate by LearningRateDecay (converging, so take finer steps); stalling divides by it (stuck, so take larger ones); the result is clamped to [min, max]. Expressing it as a scheduler means there is exactly ONE thing that writes the learning rate. Previously the inline rule and any attached scheduler both wrote it, so the adaptive branch silently overwrote the schedule on every step.

Note the direction is the opposite of reduce-on-plateau, and deliberately so: ReduceOnPlateauScheduler reduces the rate when progress stalls (assuming the step is too large to settle), whereas this raises it (assuming the step is too small to escape). Both are legitimate; they encode different beliefs about why progress stopped.

Constructors

Constructor Summary
AdaptiveFitnessScheduler(double baseLearningRate, double decay, double minLearningRate, double maxLearningRate, bool higherIsBetter) Creates an adaptive scheduler.

Methods

Method Summary
Dictionary<string, object> GetState()
double Step()
double Step(double metric)
void LoadState(Dictionary<string, object> state)
void Reset()