ExperienceReplay<T>
Models & Types · AiDotNet.ContinualLearning
Implements Experience Replay for continual learning.
Implements: IContinualLearningStrategy<T>
For Beginners
Experience Replay is one of the simplest and most effective continual learning strategies. It stores examples from previous tasks and mixes them with new task data during training, directly rehearsing old knowledge.
How It Works
How it works:
Buffer Strategies:
Advantages:
Reference: Ratcliff, R. “Connectionist models of recognition memory” (1990). Psychological Review. (Original concept); Rolnick et al. “Experience Replay for Continual Learning” (2019). NeurIPS. (Modern application)
Constructors
| Constructor | Summary |
|---|---|
ExperienceReplay(int maxBufferSize, double replayRatio, BufferStrategy<T> strategy, double lambda, int? seed) |
Initializes a new instance of the ExperienceReplay class. |
Properties
| Property | Summary |
|---|---|
BufferStrategy<T> Strategy { get; } |
Gets the buffer strategy. |
bool AccumulatesAcrossTasks { get; } |
|
double Lambda { get; set; } |
|
double ReplayRatio { get; } |
Gets the replay ratio. |
int BufferSize { get; } |
Gets the current buffer size. |
Methods
| Method | Summary |
|---|---|
T ComputeLoss(INeuralNetwork<T> network) |
|
ValueTuple<Tensor<T>, Tensor<T>> SampleMixedBatch(Tensor<T> currentInputs, Tensor<T> currentTargets, int batchSize) |
Samples a mixed batch combining current task data with replay data. |
ValueTuple<Tensor<T>, Tensor<T>> SampleReplayBatch() |
Samples a batch from the replay buffer for training. |
Vector<T> ModifyGradients(INeuralNetwork<T> network, Vector<T> gradients) |
|
void AfterTask(INeuralNetwork<T> network, ValueTuple<Tensor<T>, Tensor<T>> taskData, int taskId) |
|
void BeforeTask(INeuralNetwork<T> network, int taskId) |
|
void Reset() |