ExperienceReplay<T>

Models & Types · AiDotNet.ContinualLearning

Implements Experience Replay for continual learning.

Implements: IContinualLearningStrategy<T>

For Beginners

Experience Replay is one of the simplest and most effective continual learning strategies. It stores examples from previous tasks and mixes them with new task data during training, directly rehearsing old knowledge.

How It Works

How it works:

Buffer Strategies:

Advantages:

Reference: Ratcliff, R. “Connectionist models of recognition memory” (1990). Psychological Review. (Original concept); Rolnick et al. “Experience Replay for Continual Learning” (2019). NeurIPS. (Modern application)

Constructors

Constructor Summary
ExperienceReplay(int maxBufferSize, double replayRatio, BufferStrategy<T> strategy, double lambda, int? seed) Initializes a new instance of the ExperienceReplay class.

Properties

Property Summary
BufferStrategy<T> Strategy { get; } Gets the buffer strategy.
bool AccumulatesAcrossTasks { get; }
double Lambda { get; set; }
double ReplayRatio { get; } Gets the replay ratio.
int BufferSize { get; } Gets the current buffer size.

Methods

Method Summary
T ComputeLoss(INeuralNetwork<T> network)
ValueTuple<Tensor<T>, Tensor<T>> SampleMixedBatch(Tensor<T> currentInputs, Tensor<T> currentTargets, int batchSize) Samples a mixed batch combining current task data with replay data.
ValueTuple<Tensor<T>, Tensor<T>> SampleReplayBatch() Samples a batch from the replay buffer for training.
Vector<T> ModifyGradients(INeuralNetwork<T> network, Vector<T> gradients)
void AfterTask(INeuralNetwork<T> network, ValueTuple<Tensor<T>, Tensor<T>> taskData, int taskId)
void BeforeTask(INeuralNetwork<T> network, int taskId)
void Reset()