ElasticWeightConsolidation<T, TInput, TOutput>
Models & Types · AiDotNet.ContinualLearning.Strategies
Elastic Weight Consolidation (EWC) strategy for continual learning.
Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → ElasticWeightConsolidation<T, TInput, TOutput>
For Beginners
EWC protects important parameters from changing when learning new tasks. Think of it like a spring (elastic) that pulls parameters back toward their optimal values for previous tasks. The strength of the spring depends on how important each parameter was.
How It Works
How it works:
The Math:
EWC loss: L_total = L_task + (λ/2) * Σᵢ Fᵢ * (θᵢ - θ*ᵢ)²
Where:
Variants Supported:
Reference: Kirkpatrick et al. “Overcoming catastrophic forgetting in neural networks” (2017)
Online EWC Reference: Schwarz et al. “Progress & Compress” (2018)
Constructors
| Constructor | Summary |
|---|---|
ElasticWeightConsolidation(ILossFunction<T> lossFunction, EWCOptions<T> options) |
Initializes a new EWC strategy with custom options. |
ElasticWeightConsolidation(ILossFunction<T> lossFunction, T lambda, int numFisherSamples) |
Initializes a new EWC strategy with default options. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<Vector<T>> FisherInformation { get; } |
Gets the stored Fisher Information from previous tasks. |
IReadOnlyList<Vector<T>> OptimalParameters { get; } |
Gets the stored optimal parameters from previous tasks. |
T Lambda { get; } |
Gets the regularization strength (lambda). |
Vector<T> AccumulatedFisher { get; } |
Gets the accumulated Fisher Information (for online EWC). |
Vector<T> ConsolidatedParameters { get; } |
Gets the consolidated parameters (for online EWC). |
bool ModifiesArchitecture { get; } |
|
bool RequiresMemoryBuffer { get; } |
|
long MemoryUsageBytes { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model) |
|
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients) |
|
void FinalizeTask(IFullModel<T, TInput, TOutput> model) |
|
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData) |
|
void Reset() |