SgdGpuConfig

Options & Configuration · AiDotNet.Interfaces

Configuration for SGD (Stochastic Gradient Descent) optimizer on GPU.

Implements: IGpuOptimizerConfig

For Beginners

SGD is the simplest optimizer. It moves weights in the direction opposite to the gradient, scaled by the learning rate. Momentum helps accelerate training by accumulating velocity from past updates.

How It Works

SGD updates weights using: w = w - lr * (grad + weightDecay * w) + momentum * velocity

Constructors

Constructor Summary
SgdGpuConfig(float learningRate, float momentum, float weightDecay, int step) Creates a new SGD GPU configuration.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; }
float LearningRate { get; set; }
float Momentum { get; set; } Gets the momentum coefficient (typically 0.9).
float WeightDecay { get; set; }
int Step { get; set; }

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)