RmsPropGpuConfig

Options & Configuration · AiDotNet.Interfaces

Configuration for RMSprop optimizer on GPU.

Implements: IGpuOptimizerConfig

For Beginners

RMSprop adapts the learning rate by dividing by a running average of gradient magnitudes. This helps training be more stable when gradients vary a lot in size - common in recurrent neural networks.

How It Works

RMSprop maintains a moving average of squared gradients to normalize the gradient. This helps with non-stationary objectives and is particularly useful for RNNs.

Constructors

Constructor Summary
RmsPropGpuConfig(float learningRate, float rho, float epsilon, float weightDecay, int step) Creates a new RMSprop GPU configuration.

Properties

Property Summary
GpuOptimizerType OptimizerType { get; }
float Epsilon { get; set; } Gets the small constant for numerical stability (typically 1e-8).
float LearningRate { get; set; }
float Rho { get; set; } Gets the decay rate for the moving average (typically 0.9).
float WeightDecay { get; set; }
int Step { get; set; }

Methods

Method Summary
void ApplyUpdate(IDirectGpuBackend backend, IGpuBuffer param, IGpuBuffer gradient, GpuOptimizerState state, int size)