ScaledAdamOptimizerOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration for ScaledAdamOptimizer.
Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → ScaledAdamOptimizerOptions<T, TInput, TOutput>
How It Works
The defaults are icefall’s ScaledAdam (and GPT-SoVITS’s copy, AR/modules/optim.py).
Constructors
| Constructor | Summary |
|---|---|
ScaledAdamOptimizerOptions() |
Initializes a new instance of the ScaledAdamOptimizerOptions class with icefall’s defaults. |
Properties
| Property | Summary |
|---|---|
double Beta1 { get; set; } |
Gets or sets the momentum decay β₁ (0.9). |
double Beta2 { get; set; } |
Gets or sets the second-moment decay β₂ (0.98). |
double Epsilon { get; set; } |
Gets or sets ε (1e-8). |
double InitialLearningRate { get; set; } |
Gets or sets the learning rate (0.03). |
double ParameterMaxRms { get; set; } |
Gets or sets the largest root-mean-square a tensor’s scale learning grows it to (3). |
double ParameterMinRms { get; set; } |
Gets or sets the smallest root-mean-square a tensor’s scale learning shrinks it to (1e-5). |
double ScalarLearningRateScale { get; set; } |
Gets or sets the learning-rate factor for parameter scales and scalar parameters (0.1). |
double ScalarMax { get; set; } |
Gets or sets the largest absolute value of a scalar parameter (10). |
double? ClippingScale { get; set; } |
Gets or sets the clipping scale: gradients are clipped to this many times the median gradient norm of the last ClippingUpdatePeriod steps; null disables clipping (the default). |
int BatchSize { get; set; } |
Gets or sets the mini-batch size of Optimize (32). |
int ClippingUpdatePeriod { get; set; } |
Gets or sets how often, in steps, the clipping threshold updates (100). |
int GradientAccumulationPeriod { get; set; } |
Gets or sets the gradient accumulation period: with a period above 1, each call’s gradients are summed and the optimizer steps on calls n > 0 with n divisible by the period (counting calls from 0), as GPT-SoVITS’s s1 loop steps at `batch_id… |
int SizeUpdatePeriod { get; set; } |
Gets or sets how often, in steps, the parameter scales update (4). |