AMSGradOptimizerOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration options for the AMSGrad optimization algorithm, which is an improved variant of the Adam optimizer that addresses potential convergence issues by maintaining the maximum of past squared gradients.
Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → AMSGradOptimizerOptions<T, TInput, TOutput>
For Beginners
AMSGrad is like a smart running coach that adjusts your training pace based on your past performance. It remembers how difficult different parts of your training have been and adjusts accordingly, making sure you don’t slow down too much on challenging sections. This helps your AI model learn more efficiently by giving more attention to important patterns and less to noise in the data. Unlike some other methods, AMSGrad ensures that your learning progress never goes backward, which helps it reach better solutions.
How It Works
AMSGrad is an adaptive learning rate optimization algorithm that combines the benefits of AdaGrad and RMSProp while ensuring convergence by using a non-decreasing learning rate adjustment. It’s particularly effective for deep learning models and non-convex optimization problems.
Constructors
| Constructor | Summary |
|---|---|
AMSGradOptimizerOptions() |
Properties
| Property | Summary |
|---|---|
AMSGradBiasCorrection BiasCorrection { get; set; } |
Gets or sets which bias corrections AMSGrad applies. |
double Beta1 { get; set; } |
Gets or sets the exponential decay rate for the first moment estimates (momentum). |
double Beta2 { get; set; } |
Gets or sets the exponential decay rate for the second moment estimates (adaptive learning rates). |
double Epsilon { get; set; } |
Gets or sets a small constant added to denominators to improve numerical stability. |
double InitialLearningRate { get; set; } |
Gets or sets the initial step size used for parameter updates during optimization. |
double LearningRateDecreaseFactor { get; set; } |
Gets or sets the factor by which to decrease the learning rate when the loss is increasing or oscillating. |
double LearningRateIncreaseFactor { get; set; } |
Gets or sets the factor by which to increase the learning rate when the loss is consistently decreasing. |
double MaxLearningRate { get; set; } |
Gets or sets the maximum allowed learning rate during adaptive adjustments. |
double MinLearningRate { get; set; } |
Gets or sets the minimum allowed learning rate during adaptive adjustments. |
int BatchSize { get; set; } |
Gets or sets the batch size for mini-batch gradient descent. |