NesterovAcceleratedGradientOptimizerOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration options for the Nesterov Accelerated Gradient optimization algorithm, a momentum-based technique that improves convergence speed in gradient descent optimization.
Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → NesterovAcceleratedGradientOptimizerOptions<T, TInput, TOutput>
For Beginners
Nesterov Accelerated Gradient is a technique that helps AI models learn faster and better.
Imagine you’re trying to find the lowest point in a valley by walking downhill:
- Regular gradient descent is like always taking a step directly downhill from where you stand
- Adding momentum is like rolling a ball downhill - it picks up speed and can go faster
- Nesterov adds a clever twist: it looks ahead in the direction the ball is rolling before deciding which way is downhill
This “look-ahead” approach helps the model:
- Learn faster in most situations
- Avoid overshooting the best solution
- Navigate tricky terrain in the learning landscape
- Adapt to different types of problems
The settings in this class let you control:
- How quickly the learning rate and momentum can increase when things are going well
- How quickly they decrease when progress slows down
This adaptive behavior helps the model automatically find efficient settings as it learns, rather than requiring you to find the perfect fixed values upfront.
How It Works
Nesterov Accelerated Gradient (NAG) is an enhancement to standard gradient descent optimization that incorporates momentum with a look-ahead approach. By evaluating gradients at a position estimated by the momentum term rather than the current position, NAG provides better responsiveness to changes in the error surface. This results in faster convergence rates and improved performance, particularly in problems with high curvature or narrow valleys in the error surface. The algorithm adaptively adjusts both learning rate and momentum during training to optimize performance.
Constructors
| Constructor | Summary |
|---|---|
NesterovAcceleratedGradientOptimizerOptions() |
Initializes a new instance with the NesterovAcceleratedGradient optimizer defaults. |
NesterovAcceleratedGradientOptimizerOptions(NesterovAcceleratedGradientOptimizerOptions<T, TInput, TOutput> other) |
Initializes a new instance as a copy of other, with every setting copied. |
Properties
| Property | Summary |
|---|---|
double LearningRateDecreaseFactor { get; set; } |
Gets or sets the factor by which the learning rate is decreased when the algorithm is not making good progress. |
double LearningRateIncreaseFactor { get; set; } |
Gets or sets the factor by which the learning rate is increased when the algorithm is making good progress. |
int BatchSize { get; set; } |
Gets or sets the batch size for mini-batch gradient descent. |