RpropOptimizerOptions<T, TInput, TOutput>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Rprop (resilient backpropagation), which adapts a separate step size for every weight from the SIGN of its gradient and discards the magnitude entirely.
Inheritance: ModelOptions → OptimizationAlgorithmOptions<T, TInput, TOutput> → GradientBasedOptimizerOptions<T, TInput, TOutput> → RpropOptimizerOptions<T, TInput, TOutput>
For Beginners
Most optimizers move further when the gradient is large. Rprop ignores how large the gradient is and looks only at which direction it points. Each weight remembers its own step size: keep going the same way and the step grows by 20%; reverse direction and you must have gone too far, so the step is halved. This makes it completely immune to gradients that are vanishingly small or explosively large, which is why it was so effective on the deep-ish networks of its era.
How It Works
From Riedmiller and Braun, “A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm” (Proc. IEEE International Conference on Neural Networks, 1993, pp. 586-591, doi:10.1109/ICNN.1993.298623). The defaults below are the paper’s: eta+ = 1.2, eta- = 0.5, Delta_0 = 0.1, Delta_min = 1e-6, Delta_max = 50.
There is deliberately no learning rate here. In Rprop the per-weight step size IS the state — it grows while the gradient keeps pointing the same way and shrinks the moment it reverses — so a global learning rate would have nothing to multiply. The fused Rprop kernel takes no learning-rate argument either.
Important: Rprop requires FULL-BATCH gradients. The paper is explicit that it is a batch-learning method, and the reason is structural rather than incidental: the algorithm reads meaning into a gradient sign flip (“I overshot, halve the step”), and on mini-batches the sign flips constantly for reasons that are just sampling noise, which drives every step size down to Delta_min and stalls training. AiDotNet’s Rprop optimizer therefore computes the gradient over the entire training set on each iteration and exposes no batch size.
Constructors
| Constructor | Summary |
|---|---|
RpropOptimizerOptions() |
Initializes Rprop options with the values from the original paper. |
RpropOptimizerOptions(RpropOptimizerOptions<T, TInput, TOutput> other) |
Creates a complete copy of an existing Rprop options instance. |
Properties
| Property | Summary |
|---|---|
double EtaMinus { get; set; } |
Gets or sets the factor by which a step size shrinks when the gradient reverses sign. |
double EtaPlus { get; set; } |
Gets or sets the factor by which a step size grows when the gradient keeps its sign. |
double InitialLearningRate { get; set; } |
Gets or sets the compatibility alias for InitialStepSize. |
double InitialStepSize { get; set; } |
Gets or sets the initial per-weight update value Delta_0. |
double MaxStepSize { get; set; } |
Gets or sets the largest permitted step size Delta_max. |
double MinStepSize { get; set; } |
Gets or sets the smallest permitted step size Delta_min. |