RmsPropGpuConfig
Options & Configuration · AiDotNet.Interfaces
Configuration for RMSprop optimizer on GPU.
Implements: IGpuOptimizerConfig
For Beginners
RMSprop adapts the learning rate by dividing by a running average of gradient magnitudes. This helps training be more stable when gradients vary a lot in size - common in recurrent neural networks.
How It Works
RMSprop maintains a moving average of squared gradients to normalize the gradient. This helps with non-stationary objectives and is particularly useful for RNNs.
Constructors
| Constructor | Summary |
|---|---|
RmsPropGpuConfig(float learningRate, float rho, float epsilon, float weightDecay, int step) |
Creates a new RMSprop GPU configuration. |
Properties
| Property | Summary |
|---|---|
GpuOptimizerType OptimizerType { get; } |
|
float Epsilon { get; set; } |
Gets the small constant for numerical stability (typically 1e-8). |
float LearningRate { get; set; } |
|
float Rho { get; set; } |
Gets the decay rate for the moving average (typically 0.9). |
float WeightDecay { get; set; } |
|
int Step { get; set; } |
Methods
| Method | Summary |
|---|---|
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size) |