NagGpuConfig
Options & Configuration · AiDotNet.Interfaces
Configuration for Nesterov Accelerated Gradient (NAG) optimizer on GPU.
Implements: IGpuOptimizerConfig
For Beginners
NAG improves on regular momentum by being smarter about where to look. Instead of computing the gradient at the current position, it first moves in the direction of accumulated momentum, then computes the gradient. This “lookahead” helps it slow down before overshooting.
How It Works
NAG is a variation of momentum that looks ahead by evaluating the gradient at the “lookahead” position. This often leads to better convergence.
Constructors
| Constructor | Summary |
|---|---|
NagGpuConfig(float learningRate, float momentum, float weightDecay, int step) |
Creates a new NAG GPU configuration. |
Properties
| Property | Summary |
|---|---|
GpuOptimizerType OptimizerType { get; } |
|
float LearningRate { get; set; } |
|
float Momentum { get; set; } |
Gets the momentum coefficient (typically 0.9). |
float WeightDecay { get; set; } |
|
int Step { get; set; } |
Methods
| Method | Summary |
|---|---|
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) |