GradientBasedOptimizerOptions<T, TInput, TOutput>

Options & Configuration · AiDotNet.Models.Options

Configuration options for gradient-based optimization algorithms.

Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput>

Known derived types: ADMMOptimizerOptions<T, TInput, TOutput>, AMSGradOptimizerOptions<T, TInput, TOutput>, ASGDOptimizerOptions<T, TInput, TOutput>, AdaDeltaOptimizerOptions<T, TInput, TOutput>, AdaMaxOptimizerOptions<T, TInput, TOutput>, AdafactorOptimizerOptions<T, TInput, TOutput>, AdagradOptimizerOptions<T, TInput, TOutput>, AdamOptimizerOptions<T, TInput, TOutput>, AdamWOptimizerOptions<T, TInput, TOutput>, BFGSOptimizerOptions<T, TInput, TOutput>, ConjugateGradientOptimizerOptions<T, TInput, TOutput>, CoordinateDescentOptimizerOptions<T, TInput, TOutput>, DAdaptAdamOptimizerOptions<T, TInput, TOutput>, DFPOptimizerOptions<T, TInput, TOutput>, FTRLOptimizerOptions<T, TInput, TOutput>, GradientDescentOptimizerOptions<T, TInput, TOutput>, LAMBOptimizerOptions<T, TInput, TOutput>, LARSOptimizerOptions<T, TInput, TOutput>, LBFGSOptimizerOptions<T, TInput, TOutput>, LevenbergMarquardtOptimizerOptions<T, TInput, TOutput>, LionOptimizerOptions<T, TInput, TOutput>, MiniBatchGradientDescentOptions<T, TInput, TOutput>, MomentumOptimizerOptions<T, TInput, TOutput>, NadamOptimizerOptions<T, TInput, TOutput>, NesterovAcceleratedGradientOptimizerOptions<T, TInput, TOutput>, NewtonMethodOptimizerOptions<T, TInput, TOutput>, ProximalGradientDescentOptimizerOptions<T, TInput, TOutput>, RAdamOptimizerOptions<T, TInput, TOutput>, RootMeanSquarePropagationOptimizerOptions<T, TInput, TOutput>, RpropOptimizerOptions<T, TInput, TOutput>, ScaledAdamOptimizerOptions<T, TInput, TOutput>, ScheduleFreeAdamWOptimizerOptions<T, TInput, TOutput>, StochasticGradientDescentOptimizerOptions<T, TInput, TOutput>, TrustRegionOptimizerOptions<T, TInput, TOutput>

For Beginners

Imagine you’re in a hilly landscape and want to find the lowest point. Gradient-based optimization is like always walking downhill in the steepest direction until you can’t go any lower. The “gradient” is simply the direction of the steepest slope at your current position.

How It Works

Gradient-based optimizers are algorithms that find the minimum or maximum of a function by following the direction of steepest descent or ascent (the gradient).

These algorithms are fundamental to training many machine learning models, including neural networks, linear regression, and logistic regression.

This class inherits from OptimizationAlgorithmOptions, which means it includes all the base configuration options for optimization algorithms plus any additional options specific to gradient-based methods.

Constructors

Constructor Summary
GradientBasedOptimizerOptions() Creates gradient optimizer options with the documented defaults.
GradientBasedOptimizerOptions(GradientBasedOptimizerOptions<T, TInput, TOutput> other) Copies every shared and gradient-specific optimizer setting.

Properties

Property Summary
GradientClippingMethod GradientClippingMethod { get; set; } Gets or sets the gradient clipping method to use.
IDataSampler DataSampler { get; set; } Gets or sets the optional data sampler for advanced sampling strategies during batch creation.
IGradientCache<T> GradientCache { get; set; } Gets or sets the gradient cache to use for storing and retrieving computed gradients.
ILearningRateScheduler LearningRateScheduler { get; set; } Gets or sets the learning rate scheduler to use during training.
ILossFunction<T> LossFunction { get; set; } Gets or sets the loss function to use for evaluating model performance.
IRegularization<T, TInput, TOutput> Regularization { get; set; } Gets or sets the regularization method to use for preventing overfitting.
RegularizationOrder RegularizationOrder { get; set; } Gets or sets which side of gradient clipping the optimizer’s regularization (L2 and the others) is added on.
SchedulerStepMode SchedulerStepMode { get; set; } Gets or sets when the learning rate scheduler should be stepped.
bool DropLastBatch { get; set; } Gets or sets whether to drop the last incomplete batch.
bool EnableGradientClipping { get; set; } Gets or sets whether gradient clipping is enabled.
bool ShuffleData { get; set; } Gets or sets whether to shuffle data at the beginning of each epoch.
bool UseStrongWolfeLineSearch { get; set; } Gets or sets whether the function-minimization line search enforces the strong Wolfe conditions rather than sufficient decrease alone.
bool UseTrainingLossAsFitness { get; set; } Gets or sets whether the optimizer uses the running mean of the per-mini-batch training loss as each epoch’s fitness, instead of running a separate full-dataset forward pass every epoch.
double ArmijoConstant { get; set; } Gets or sets the Armijo sufficient-decrease constant used by the line search.
double LineSearchContractionFactor { get; set; } Gets or sets the factor by which the trial step is shrunk on each line search backtrack.
double LineSearchFallbackStep { get; set; } Gets or sets the step length used when the line search fails to find an acceptable step.
double LineSearchMaxStep { get; set; } Gets or sets the largest step length the strong Wolfe search will consider.
double MaxGradientNorm { get; set; } Gets or sets the maximum gradient norm for norm-based clipping.
double MaxGradientValue { get; set; } Gets or sets the maximum gradient value for value-based clipping.
double WolfeCurvatureConstant { get; set; } Gets or sets the curvature constant c2 in the strong Wolfe conditions.
int LineSearchMaxSteps { get; set; } Gets or sets the maximum number of backtracking steps in the line search.
int LineSearchMaxZoomSteps { get; set; } Gets or sets how many refinement steps the strong Wolfe search may take once it has bracketed an acceptable step length.