ScaledAdamOptimizerOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.Models.Options

Configuration for ScaledAdamOptimizer.

Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → ScaledAdamOptimizerOptions<T, TInput, TOutput>

How It Works

The defaults are icefall’s ScaledAdam (and GPT-SoVITS’s copy, AR/modules/optim.py).

Constructors

Constructor Summary
ScaledAdamOptimizerOptions() Initializes a new instance of the ScaledAdamOptimizerOptions class with icefall’s defaults.

Properties

Property Summary
double Beta1 { get; set; } Gets or sets the momentum decay β₁ (0.9).
double Beta2 { get; set; } Gets or sets the second-moment decay β₂ (0.98).
double Epsilon { get; set; } Gets or sets ε (1e-8).
double InitialLearningRate { get; set; } Gets or sets the learning rate (0.03).
double ParameterMaxRms { get; set; } Gets or sets the largest root-mean-square a tensor’s scale learning grows it to (3).
double ParameterMinRms { get; set; } Gets or sets the smallest root-mean-square a tensor’s scale learning shrinks it to (1e-5).
double ScalarLearningRateScale { get; set; } Gets or sets the learning-rate factor for parameter scales and scalar parameters (0.1).
double ScalarMax { get; set; } Gets or sets the largest absolute value of a scalar parameter (10).
double? ClippingScale { get; set; } Gets or sets the clipping scale: gradients are clipped to this many times the median gradient norm of the last ClippingUpdatePeriod steps; null disables clipping (the default).
int BatchSize { get; set; } Gets or sets the mini-batch size of Optimize (32).
int ClippingUpdatePeriod { get; set; } Gets or sets how often, in steps, the clipping threshold updates (100).
int GradientAccumulationPeriod { get; set; } Gets or sets the gradient accumulation period: with a period above 1, each call’s gradients are summed and the optimizer steps on calls n > 0 with n divisible by the period (counting calls from 0), as GPT-SoVITS’s s1 loop steps at `batch_id…
int SizeUpdatePeriod { get; set; } Gets or sets how often, in steps, the parameter scales update (4).