LearningRateSchedulerFactory

Helpers & Utilities · AiDotNet.LearningRateSchedulers

Factory for creating learning rate schedulers with common configurations.

For Beginners

Instead of manually configuring schedulers with many parameters, you can use this factory to create schedulers optimized for specific scenarios. For example, CreateForTransformer() creates a scheduler tuned for transformer model training, with warmup and linear decay that works well for attention-based models.

How It Works

This factory provides convenient methods for creating pre-configured learning rate schedulers for common use cases. It simplifies scheduler creation and provides sensible defaults for various training scenarios.

Methods

Method Summary
ILearningRateScheduler Create(LearningRateSchedulerType type, double baseLearningRate, int totalSteps) Creates a scheduler based on type enum.
ILearningRateScheduler CreateAdaptive(double baseLearningRate, double factor, int patience, double minLr) Creates a learning rate scheduler that adapts based on validation loss.
ILearningRateScheduler CreateForCNN(double baseLearningRate, int stepSize, double gamma) Creates a learning rate scheduler for typical CNN training.
ILearningRateScheduler CreateForFineTuning(double baseLearningRate) Creates a learning rate scheduler for fine-tuning pre-trained models.
ILearningRateScheduler CreateForLongTraining(double baseLearningRate, int totalEpochs, double etaMin) Creates a learning rate scheduler for long training runs.
ILearningRateScheduler CreateForSuperConvergence(double maxLearningRate, int totalSteps, double pctStart) Creates a learning rate scheduler for super-convergence training.
ILearningRateScheduler CreateForTransformer(double baseLearningRate, int warmupSteps, int totalSteps) Creates a learning rate scheduler for transformer training.
ILearningRateScheduler CreateWithWarmRestarts(double baseLearningRate, int t0, int tMult, double etaMin) Creates a learning rate scheduler with warm restarts.