UniformReplayBuffer<T, TState, TAction>
Models & Types · AiDotNet.ReinforcementLearning.ReplayBuffers
A replay buffer that samples experiences uniformly at random.
Implements: IReplayBuffer<T, TState, TAction>
For Beginners
This replay buffer treats all experiences equally - it’s like having a bag of memories and pulling out random ones to learn from. When the buffer is full, the oldest memories get replaced with new ones.
Key Properties:
- Uniform Sampling: Every experience has an equal chance of being picked
- Circular Buffer: Old experiences are automatically removed when capacity is reached
- No Prioritization: Unlike prioritized replay, doesn’t favor “important” experiences
When to Use:
- Good starting point for most RL algorithms
- Works well when all experiences are roughly equally valuable
- Simpler and faster than prioritized variants
How It Works
This is the standard replay buffer used in algorithms like DQN. Experiences are stored in a circular buffer and sampled uniformly at random for training. All experiences have an equal probability of being selected, regardless of their importance or recency.
Constructors
| Constructor | Summary |
|---|---|
UniformReplayBuffer() |
|
UniformReplayBuffer(int capacity, int? seed) |
Initializes a new instance of the UniformReplayBuffer class. |
Properties
| Property | Summary |
|---|---|
int Capacity { get; } |
|
int Count { get; } |
Methods
| Method | Summary |
|---|---|
List<Experience<T, TState, TAction>> Sample(int batchSize) |
|
List<Experience<T, TState, TAction>> SampleSequence(int length) |
Samples up to length TEMPORALLY-CONSECUTIVE experiences starting at a random valid position — the trajectory window MuZero’s K-step unroll needs (not independent transitions). |
ValueTuple<List<Experience<T, TState, TAction>>, List<int>> SampleWithIndices(int batchSize) |
Samples a batch of experiences with their buffer indices. |
bool CanSample(int batchSize) |
|
void Add(Experience<T, TState, TAction> experience) |
|
void Clear() |