AdamGpuConfig

Options & Configuration · AiDotNet.Interfaces

Configuration for Adam optimizer on GPU.

Implements: IGpuOptimizerConfig

For Beginners

Adam is one of the most popular optimizers. It adapts the learning rate for each parameter based on:

  • First moment (mean of gradients) - like momentum
  • Second moment (variance of gradients) - adapts to gradient magnitude

This typically leads to faster convergence than plain SGD.

How It Works

Adam maintains moving averages of gradients (m) and squared gradients (v), with bias correction for the initial steps.

Constructors

Constructor Summary
AdamGpuConfig(float learningRate, float beta1, float beta2, float epsilon, float weightDecay, int step) Creates a new Adam GPU configuration.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; }
float Beta1 { get; set; } Gets the exponential decay rate for the first moment estimates (typically 0.9).
float Beta2 { get; set; } Gets the exponential decay rate for the second moment estimates (typically 0.999).
float Epsilon { get; set; } Gets the small constant for numerical stability (typically 1e-8).
float LearningRate { get; set; }
float WeightDecay { get; set; }
int Step { get; set; }

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)