UniformReplayBuffer<T, TState, TAction>

Models & Types · AiDotNet.ReinforcementLearning.ReplayBuffers

A replay buffer that samples experiences uniformly at random.

Implements: IReplayBuffer<T, TState, TAction>

For Beginners

This replay buffer treats all experiences equally - it’s like having a bag of memories and pulling out random ones to learn from. When the buffer is full, the oldest memories get replaced with new ones.

Key Properties:

  • Uniform Sampling: Every experience has an equal chance of being picked
  • Circular Buffer: Old experiences are automatically removed when capacity is reached
  • No Prioritization: Unlike prioritized replay, doesn’t favor “important” experiences

When to Use:

  • Good starting point for most RL algorithms
  • Works well when all experiences are roughly equally valuable
  • Simpler and faster than prioritized variants

How It Works

This is the standard replay buffer used in algorithms like DQN. Experiences are stored in a circular buffer and sampled uniformly at random for training. All experiences have an equal probability of being selected, regardless of their importance or recency.

Constructors

Constructor Summary
UniformReplayBuffer()
UniformReplayBuffer(int capacity, int? seed) Initializes a new instance of the UniformReplayBuffer class.

Properties

Property Summary
int Capacity { get; }
int Count { get; }

Methods

Method Summary
List<Experience<T, TState, TAction>> Sample(int batchSize)
List<Experience<T, TState, TAction>> SampleSequence(int length) Samples up to length TEMPORALLY-CONSECUTIVE experiences starting at a random valid position — the trajectory window MuZero’s K-step unroll needs (not independent transitions).
ValueTuple<List<Experience<T, TState, TAction>>, List<int>> SampleWithIndices(int batchSize) Samples a batch of experiences with their buffer indices.
bool CanSample(int batchSize)
void Add(Experience<T, TState, TAction> experience)
void Clear()