RLTrainingOptions<T>

Options & Configuration · AiDotNet.Configuration

Configuration options for reinforcement learning training loops via AiModelBuilder.

For Beginners

Reinforcement learning trains an agent through trial and error in an environment. This options class lets you customize every aspect of that training process:

  • How many episodes to run
  • How to explore vs exploit
  • How to store and sample experiences
  • When to receive progress updates

Quick Start Example:

How It Works

This class provides comprehensive configuration for RL training loops, following industry-standard patterns from libraries like Stable-Baselines3, RLlib, and CleanRL.

Note: This class is for configuring the training loop (episodes, steps, callbacks). For agent-specific options (learning rate, discount factor), see each agent’s options class.

Constructors

Constructor Summary
RLTrainingOptions()

Properties

Property Summary
Action OnTrainingStart { get; set; } Gets or sets the callback invoked when training starts.
Action<RLEpisodeMetrics<T>> OnEpisodeComplete { get; set; } Gets or sets the callback invoked after each episode completes.
Action<RLStepMetrics<T>> OnStepComplete { get; set; } Gets or sets the callback invoked after each training step.
Action<RLTrainingSummary<T>> OnTrainingComplete { get; set; } Gets or sets the callback invoked when training ends.
ExplorationScheduleConfig<T> ExplorationSchedule { get; set; } Gets or sets the exploration schedule configuration.
IEnvironment<T> Environment { get; set; } Gets or sets the environment for the agent to interact with.
IExplorationStrategy<T> ExplorationStrategy { get; set; } Gets or sets the optional exploration strategy to use during training.
IIntrinsicRewardModule<T> IntrinsicRewardModule { get; set; } Gets or sets an optional intrinsic-reward (curiosity) module.
IReplayBuffer<T, Vector<T>, Vector<T>> ReplayBuffer { get; set; } Gets or sets the optional replay buffer for experience storage.
PrioritizedReplayConfig<T> PrioritizedReplayConfig { get; set; } Gets or sets the prioritized replay configuration.
RLCheckpointConfig CheckpointConfig { get; set; } Gets or sets the checkpoint configuration for saving models during training.
RLEarlyStoppingConfig<T> EarlyStoppingConfig { get; set; } Gets or sets the early stopping configuration.
RLEvaluationConfig EvaluationConfig { get; set; } Gets or sets the evaluation configuration for assessing agent performance during training.
RewardClippingConfig<T> RewardClipping { get; set; } Gets or sets the reward clipping bounds.
TargetNetworkConfig<T> TargetNetworkConfig { get; set; } Gets or sets the target network configuration for DQN-family algorithms.
bool NormalizeObservations { get; set; } Gets or sets whether to normalize observations.
bool NormalizeRewards { get; set; } Gets or sets whether to normalize rewards.
bool UsePrioritizedReplay { get; set; } Gets or sets whether to use prioritized experience replay.
double IntrinsicRewardWeight { get; set; } Gets or sets the weight applied to the intrinsic reward before it is added to the extrinsic reward.
int BatchSize { get; set; } Gets or sets the batch size for sampling from the replay buffer.
int Episodes { get; set; } Gets or sets the number of episodes to train for.
int GradientSteps { get; set; } Gets or sets the number of gradient steps per training update.
int LogFrequency { get; set; } Gets or sets how often to log progress (every N episodes).
int MaxStepsPerEpisode { get; set; } Gets or sets the maximum steps per episode to prevent infinite loops.
int TrainFrequency { get; set; } Gets or sets the frequency of training updates (every N steps).
int WarmupSteps { get; set; } Gets or sets the number of initial random steps before training begins.
int? Seed { get; set; } Gets or sets the random seed for reproducibility.

Methods

Method Summary
RLTrainingOptions<T> Default(IEnvironment<T> environment) Creates default options with sensible values for most use cases.