AdagradGpuConfig

Options & Configuration · AiDotNet.Interfaces

Configuration for Adagrad optimizer on GPU.

Implements: IGpuOptimizerConfig

For Beginners

Adagrad is good for sparse data because it gives larger updates to infrequent parameters and smaller updates to frequent ones. However, the accumulated squared gradients can make learning rate too small eventually.

How It Works

Adagrad accumulates squared gradients over all time, providing automatic learning rate adaptation. Parameters with frequently occurring features get smaller learning rates.

Constructors

Constructor Summary
AdagradGpuConfig(float learningRate, float epsilon, float weightDecay, int step) Creates a new Adagrad GPU configuration.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; }
float Epsilon { get; set; } Gets the small constant for numerical stability (typically 1e-8).
float LearningRate { get; set; }
float WeightDecay { get; set; }
int Step { get; set; }

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)