GradientEpisodicMemory<T, TInput, TOutput>
Models & Types · AiDotNet.ContinualLearning.Strategies
Gradient Episodic Memory (GEM) strategy for continual learning with gradient constraints.
Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → GradientEpisodicMemory<T, TInput, TOutput>
Implements: IGradientConstraintStrategy<T, TInput, TOutput>
For Beginners
GEM prevents forgetting by ensuring that parameter updates for new tasks don’t increase the loss on any previous task. It does this by:
- Storing representative examples from each task (episodic memory)
- Computing reference gradients for each previous task
- Projecting the current gradient to satisfy all task constraints
How It Works
How Gradient Projection Works: If the current gradient g would increase loss on task k (i.e., g · g_k < 0), we project g to the closest gradient g’ such that g’ · g_k ≥ 0 for all previous tasks. This is a Quadratic Programming (QP) problem.
GEM vs A-GEM:
- GEM: Checks all task constraints individually. Strong guarantees but O(t) complexity.
- A-GEM: Uses average reference gradient. O(1) but weaker guarantees.
Advantages:
- Strong theoretical guarantees (never increases loss on previous tasks)
- Works with limited memory
- Applicable to any gradient-based optimizer
Disadvantages:
- Requires solving QP for gradient projection
- Needs access to gradients (not just loss values)
- Memory scales linearly with number of tasks
References:
- Lopez-Paz & Ranzato “Gradient Episodic Memory for Continual Learning” (NeurIPS 2017)
- Chaudhry et al. “Efficient Lifelong Learning with A-GEM” (ICLR 2019)
Constructors
| Constructor | Summary |
|---|---|
GradientEpisodicMemory(ILossFunction<T> lossFunction, GEMOptions<T> options) |
Initializes a new Gradient Episodic Memory strategy. |
Properties
| Property | Summary |
|---|---|
ExperienceReplayBuffer<T, TInput, TOutput> MemoryBuffer { get; } |
Gets the episodic memory buffer. |
bool ModifiesArchitecture { get; } |
|
bool RequiresMemoryBuffer { get; } |
|
int MaxExamples { get; } |
|
int StoredExampleCount { get; } |
|
int StoredGradientCount { get; } |
|
long MemoryUsageBytes { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, object> GetMetrics() |
|
IReadOnlyList<ValueTuple<TInput, TOutput, int>> SampleExamples(int batchSize) |
|
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model) |
|
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients) |
|
Vector<T> ProjectGradient(Vector<T> gradient) |
|
bool ViolatesConstraint(Vector<T> gradient) |
|
void ClearMemory() |
|
void FinalizeTask(IFullModel<T, TInput, TOutput> model) |
|
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData) |
|
void Reset() |
|
void StoreTaskExamples(IDataset<T, TInput, TOutput> taskData) |
|
void StoreTaskGradient(Vector<T> taskGradient) |