IReplayBuffer<T, TState, TAction>
Interfaces · AiDotNet.ReinforcementLearning.ReplayBuffers
Interface for experience replay buffers used in reinforcement learning.
Known implementations: UniformReplayBuffer<T, TState, TAction>
For Beginners
A replay buffer is like a memory bank for the agent. Instead of learning only from the most recent experience, the agent stores experiences and learns from random samples of past experiences. This makes learning more stable and efficient.
Think of it like studying for an exam:
- You don’t just study the most recent lesson
- You review random material from throughout the course
- This helps you learn connections between different topics
- And prevents forgetting older material
Common Buffer Types:
- Uniform: All experiences sampled with equal probability
- Prioritized: Important experiences (big errors) sampled more often
How It Works
Experience replay is a technique where the agent stores past experiences and learns from them multiple times. This breaks temporal correlations and improves sample efficiency. The replay buffer stores experience tuples (state, action, reward, next_state, done) and provides random sampling for training.
Properties
| Property | Summary |
|---|---|
int Capacity { get; } |
Gets the maximum capacity of the buffer. |
int Count { get; } |
Gets the current number of experiences in the buffer. |
Methods
| Method | Summary |
|---|---|
List<Experience<T, TState, TAction>> Sample(int batchSize) |
Samples a batch of experiences from the buffer. |
bool CanSample(int batchSize) |
Checks if the buffer has enough experiences to sample a batch. |
void Add(Experience<T, TState, TAction> experience) |
Adds an experience to the buffer. |
void Clear() |
Clears all experiences from the buffer. |