GradientEpisodicMemory<T, TInput, TOutput>

Models & Types · AiDotNet.ContinualLearning.Strategies

Gradient Episodic Memory (GEM) strategy for continual learning with gradient constraints.

Inheritance: ContinualLearningStrategyBase<T, TInput, TOutput> → GradientEpisodicMemory<T, TInput, TOutput>

Implements: IGradientConstraintStrategy<T, TInput, TOutput>

For Beginners

GEM prevents forgetting by ensuring that parameter updates for new tasks don’t increase the loss on any previous task. It does this by:

  1. Storing representative examples from each task (episodic memory)
  2. Computing reference gradients for each previous task
  3. Projecting the current gradient to satisfy all task constraints

How It Works

How Gradient Projection Works: If the current gradient g would increase loss on task k (i.e., g · g_k < 0), we project g to the closest gradient g’ such that g’ · g_k ≥ 0 for all previous tasks. This is a Quadratic Programming (QP) problem.

GEM vs A-GEM:

  • GEM: Checks all task constraints individually. Strong guarantees but O(t) complexity.
  • A-GEM: Uses average reference gradient. O(1) but weaker guarantees.

Advantages:

  • Strong theoretical guarantees (never increases loss on previous tasks)
  • Works with limited memory
  • Applicable to any gradient-based optimizer

Disadvantages:

  • Requires solving QP for gradient projection
  • Needs access to gradients (not just loss values)
  • Memory scales linearly with number of tasks

References:

  • Lopez-Paz & Ranzato “Gradient Episodic Memory for Continual Learning” (NeurIPS 2017)
  • Chaudhry et al. “Efficient Lifelong Learning with A-GEM” (ICLR 2019)

Constructors

Constructor Summary
GradientEpisodicMemory(ILossFunction<T> lossFunction, GEMOptions<T> options) Initializes a new Gradient Episodic Memory strategy.

Properties

Property Summary
ExperienceReplayBuffer<T, TInput, TOutput> MemoryBuffer { get; } Gets the episodic memory buffer.
bool ModifiesArchitecture { get; }
bool RequiresMemoryBuffer { get; }
int MaxExamples { get; }
int StoredExampleCount { get; }
int StoredGradientCount { get; }
long MemoryUsageBytes { get; }
string Name { get; }

Methods

Method Summary
IReadOnlyDictionary<string, object> GetMetrics()
IReadOnlyList<ValueTuple<TInput, TOutput, int>> SampleExamples(int batchSize)
T ComputeRegularizationLoss(IFullModel<T, TInput, TOutput> model)
Vector<T> AdjustGradients(IFullModel<T, TInput, TOutput> model, Vector<T> gradients)
Vector<T> ProjectGradient(Vector<T> gradient)
bool ViolatesConstraint(Vector<T> gradient)
void ClearMemory()
void FinalizeTask(IFullModel<T, TInput, TOutput> model)
void PrepareForTask(IFullModel<T, TInput, TOutput> model, IDataset<T, TInput, TOutput> taskData)
void Reset()
void StoreTaskExamples(IDataset<T, TInput, TOutput> taskData)
void StoreTaskGradient(Vector<T> taskGradient)