AdagradGpuConfig
Options & Configuration · AiDotNet.Interfaces
Configuration for Adagrad optimizer on GPU.
Implements: IGpuOptimizerConfig
For Beginners
Adagrad is good for sparse data because it gives larger updates to infrequent parameters and smaller updates to frequent ones. However, the accumulated squared gradients can make learning rate too small eventually.
How It Works
Adagrad accumulates squared gradients over all time, providing automatic learning rate adaptation. Parameters with frequently occurring features get smaller learning rates.
Constructors
| Constructor | Summary |
|---|---|
AdagradGpuConfig(float learningRate, float epsilon, float weightDecay, int step) |
Creates a new Adagrad GPU configuration. |
Properties
| Property | Summary |
|---|---|
GpuOptimizerType OptimizerType { get; } |
|
float Epsilon { get; set; } |
Gets the small constant for numerical stability (typically 1e-8). |
float LearningRate { get; set; } |
|
float WeightDecay { get; set; } |
|
int Step { get; set; } |
Methods
| Method | Summary |
|---|---|
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) |