ExpectedGradientLength<T, TInput, TOutput>
Models & Types · AiDotNet.ContinualLearning.Strategies
Expected Gradient Length (EGL) strategy for continual learning.
Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → ExpectedGradientLength<T, TInput, TOutput>
For Beginners
EGL protects important parameters by measuring how much each parameter affects the output when training on a task. Parameters with larger expected gradient lengths are considered more important.
How It Works
How it works:
The Math:
EGL loss: L_total = L_task + (λ/2) * Σᵢ Ωᵢ * (θᵢ - θ*ᵢ)²
Where Ωᵢ = E[|∂L/∂θᵢ|] is the expected gradient length for parameter i
Comparison to Other Methods:
EGL is simpler than EWC and provides a direct measure of how much each parameter contributes to the loss during training.
Constructors
| Constructor | Summary |
|---|---|
ExpectedGradientLength(ILossFunction<T> lossFunction, EGLOptions<T> options) |
Initializes a new EGL strategy with custom options. |
ExpectedGradientLength(ILossFunction<T> lossFunction, T lambda, int numSamples) |
Initializes a new EGL strategy with default options. |
Properties
| Property | Summary |
|---|---|
T Lambda { get; } |
Gets the regularization strength (lambda). |
Vector<T> Importance { get; } |
Gets the accumulated importance weights. |
Vector<T> PreviousParameters { get; } |
Gets the stored parameters from previous tasks. |
bool ModifiesArchitecture { get; } |
|
bool RequiresMemoryBuffer { get; } |
|
long MemoryUsageBytes { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model) |
|
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients) |
|
void FinalizeTask(IFullModel<T, TInput, TOutput> model) |
|
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData) |
|
void Reset() |