MetaSGDLearningRateInitialization
Enums · AiDotNet.MetaLearning.Options
Learning rate initialization strategies for Meta-SGD.
How It Works
Different initialization strategies can affect how quickly Meta-SGD converges and the quality of the learned per-parameter optimizers.
Fields
| Field | Summary |
|---|---|
MetaSGDLearningRateInitialization LayerBased |
Initialize learning rates based on layer depth. |
MetaSGDLearningRateInitialization MagnitudeBased |
Initialize learning rates based on parameter magnitudes. |
MetaSGDLearningRateInitialization Random |
Initialize learning rates uniformly at random within LearningRateInitRange. |
MetaSGDLearningRateInitialization Uniform |
Initialize all learning rates to the same value (InnerLearningRate). |
MetaSGDLearningRateInitialization Xavier |
Initialize learning rates using Xavier-style initialization. |
int value__ |