NagGpuConfig

Options & Configuration · AiDotNet.Interfaces

Configuration for Nesterov Accelerated Gradient (NAG) optimizer on GPU.

Implements: IGpuOptimizerConfig

For Beginners

NAG improves on regular momentum by being smarter about where to look. Instead of computing the gradient at the current position, it first moves in the direction of accumulated momentum, then computes the gradient. This “lookahead” helps it slow down before overshooting.

How It Works

NAG is a variation of momentum that looks ahead by evaluating the gradient at the “lookahead” position. This often leads to better convergence.

Constructors

Constructor Summary
NagGpuConfig(float learningRate, float momentum, float weightDecay, int step) Creates a new NAG GPU configuration.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; }
float LearningRate { get; set; }
float Momentum { get; set; } Gets the momentum coefficient (typically 0.9).
float WeightDecay { get; set; }
int Step { get; set; }

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)