IGpuOptimizerConfig
Interfaces · AiDotNet.Interfaces
Configuration for GPU-resident optimizer updates.
Known implementations: AdagradGpuConfig, AdamGpuConfig, AdamWGpuConfig, LambGpuConfig, LarsGpuConfig, NagGpuConfig, RmsPropGpuConfig, SgdGpuConfig
For Beginners
When training on GPU, the weights need to be updated using an optimizer (like SGD or Adam). This configuration tells the GPU exactly how to update the weights - with what learning rate, momentum, etc.
How It Works
This interface allows layers to receive optimizer-specific configuration for GPU parameter updates. Different optimizer types (SGD, Adam, etc.) have different implementations with their specific hyperparameters.
Properties
| Property | Summary |
|---|---|
GpuOptimizerType OptimizerType { get; } |
Gets the type of optimizer (SGD, Adam, AdamW, etc.). |
float LearningRate { get; } |
Gets the learning rate for parameter updates. |
float WeightDecay { get; } |
Gets the weight decay (L2 regularization) coefficient. |
int Step { get; } |
Gets the current optimization step (used for bias correction in Adam-family optimizers). |
Methods
| Method | Summary |
|---|---|
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) |
Applies the optimizer update to the given parameter buffer using its gradient. |