IGpuOptimizerConfig

Interfaces · AiDotNet.Interfaces

Configuration for GPU-resident optimizer updates.

Known implementations: AdagradGpuConfig, AdamGpuConfig, AdamWGpuConfig, LambGpuConfig, LarsGpuConfig, NagGpuConfig, RmsPropGpuConfig, SgdGpuConfig

For Beginners

When training on GPU, the weights need to be updated using an optimizer (like SGD or Adam). This configuration tells the GPU exactly how to update the weights - with what learning rate, momentum, etc.

How It Works

This interface allows layers to receive optimizer-specific configuration for GPU parameter updates. Different optimizer types (SGD, Adam, etc.) have different implementations with their specific hyperparameters.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; } Gets the type of optimizer (SGD, Adam, AdamW, etc.).
float LearningRate { get; } Gets the learning rate for parameter updates.
float WeightDecay { get; } Gets the weight decay (L2 regularization) coefficient.
int Step { get; } Gets the current optimization step (used for bias correction in Adam-family optimizers).

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) Applies the optimizer update to the given parameter buffer using its gradient.