AdamOptimizerOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration options for the Adam optimization algorithm, which combines the benefits of AdaGrad and RMSProp.
Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → AdamOptimizerOptions<T, TInput, TOutput>
Known derived types: Adam8BitOptimizerOptions<T, TInput, TOutput>
For Beginners
Adam is like a smart learning assistant that remembers both the direction (momentum) and the size of previous steps. It automatically adjusts how big each step should be for each parameter, making it easier to train models without having to manually tune the learning rate. Adam is often a good default choice for many machine learning problems.
How It Works
Adam (Adaptive Moment Estimation) is a popular optimization algorithm that computes adaptive learning rates for each parameter. It stores both an exponentially decaying average of past gradients (first moment) and past squared gradients (second moment).
Constructors
| Constructor | Summary |
|---|---|
AdamOptimizerOptions() |
Default ctor — overrides the base EnableGradientClipping default from false to true with MaxGradientNorm at the canonical PyTorch transformer-training value of 1.0. |
AdamOptimizerOptions(AdamOptimizerOptions<T, TInput, TOutput> other) |
Copy constructor — clones every property declared on this options class plus the base-class gradient-clipping settings used here. |
Properties
| Property | Summary |
|---|---|
AdamAnomalyGuardMode AnomalyGuardMode { get; set; } |
Gets or sets the policy for the PyTorch GradScaler-style anomaly guard that skips an Adam step when any gradient contains NaN or Inf. |
bool UseAMSGrad { get; set; } |
Gets or sets whether to use the AMSGrad variant of Adam. |
bool UseAdaptiveBetas { get; set; } |
Gets or sets whether to automatically adjust the Beta parameters during training. |
double Beta1 { get; set; } |
Gets or sets the exponential decay rate for the first moment estimates. |
double Beta2 { get; set; } |
Gets or sets the exponential decay rate for the second moment estimates. |
double Epsilon { get; set; } |
Gets or sets a small constant added to denominators to prevent division by zero. |
double InitialLearningRate { get; set; } |
Gets or sets the initial learning rate for the Adam optimizer. |
double MaxBeta1 { get; set; } |
Gets or sets the maximum allowed value for Beta1. |
double MaxBeta2 { get; set; } |
Gets or sets the maximum allowed value for Beta2. |
double MinBeta1 { get; set; } |
Gets or sets the minimum allowed value for Beta1. |
double MinBeta2 { get; set; } |
Gets or sets the minimum allowed value for Beta2. |
int BatchSize { get; set; } |
Gets or sets the batch size for mini-batch gradient descent. |