IRLDataLoader<T>

Interfaces · AiDotNet.Interfaces

Interface for data loaders that provide experience data for reinforcement learning.

Implements: IBatchIterable<Experience<T, Vector<T>, Vector<T>>>, IDataLoader<T>

Known implementations: RLDataLoaderBase<T>

For Beginners

Reinforcement learning is learning through trial and error.

How it works:

  • An agent takes actions in an environment
  • The environment returns rewards and new states
  • The agent learns to maximize total rewards

Example: Game Playing

  • Environment: The game (e.g., CartPole, Atari, Chess)
  • State: What the agent sees (game screen, piece positions)
  • Action: What the agent does (move left, jump, place piece)
  • Reward: Score or outcome (+1 for winning, -1 for losing)

This data loader:

  • Runs episodes in the environment
  • Collects experience tuples (state, action, reward, next_state, done)
  • Stores them in a replay buffer for training
  • Provides batches of experiences for learning

How It Works

This interface is for reinforcement learning scenarios where an agent interacts with an environment to collect experience data. The loader manages:

  • Environment interactions (stepping through episodes)
  • Experience collection and storage
  • Replay buffer management for batch sampling

Properties

Property Summary
IEnvironment<T> Environment { get; } Gets the environment that the agent interacts with.
IReplayBuffer<T, Vector<T>, Vector<T>> ReplayBuffer { get; } Gets the replay buffer used for storing and sampling experiences.
bool Verbose { get; } Gets whether to print training progress to console.
int CurrentEpisode { get; } Gets the current episode number (0-indexed).
int Episodes { get; } Gets the total number of episodes to run during training.
int MaxStepsPerEpisode { get; } Gets the maximum number of steps per episode (prevents infinite episodes).
int MinExperiencesBeforeTraining { get; } Gets the minimum number of experiences required before training can begin.
int TotalSteps { get; } Gets the total number of steps taken across all episodes.

Methods

Method Summary
EpisodeResult<T> RunEpisode(IRLAgent<T> agent) Runs a single episode and collects experiences.
IReadOnlyList<EpisodeResult<T>> RunEpisodes(int numEpisodes, IRLAgent<T> agent) Runs multiple episodes and collects experiences.
IReadOnlyList<Experience<T, Vector<T>, Vector<T>>> SampleBatch(int batchSize) Samples a batch of experiences from the replay buffer.
bool CanTrain(int batchSize) Checks if there are enough experiences to begin training.
void AddExperience(Experience<T, Vector<T>, Vector<T>> experience) Adds an experience to the replay buffer.
void ResetTraining() Resets the data loader state (clears buffer, resets counters).
void SetSeed(int seed) Sets the random seed for reproducible training.