AdamGpuConfig
Options & Configuration · AiDotNet.Interfaces
Configuration for Adam optimizer on GPU.
Implements: IGpuOptimizerConfig
For Beginners
Adam is one of the most popular optimizers. It adapts the learning rate for each parameter based on:
- First moment (mean of gradients) - like momentum
- Second moment (variance of gradients) - adapts to gradient magnitude
This typically leads to faster convergence than plain SGD.
How It Works
Adam maintains moving averages of gradients (m) and squared gradients (v), with bias correction for the initial steps.
Constructors
| Constructor | Summary |
|---|---|
AdamGpuConfig(float learningRate, float beta1, float beta2, float epsilon, float weightDecay, int step) |
Creates a new Adam GPU configuration. |
Properties
| Property | Summary |
|---|---|
GpuOptimizerType OptimizerType { get; } |
|
float Beta1 { get; set; } |
Gets the exponential decay rate for the first moment estimates (typically 0.9). |
float Beta2 { get; set; } |
Gets the exponential decay rate for the second moment estimates (typically 0.999). |
float Epsilon { get; set; } |
Gets the small constant for numerical stability (typically 1e-8). |
float LearningRate { get; set; } |
|
float WeightDecay { get; set; } |
|
int Step { get; set; } |
Methods
| Method | Summary |
|---|---|
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) |