RainbowDQNOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for Rainbow DQN agent.

Inheritance: ModelOptions → ReinforcementLearningOptions<T> → RainbowDQNOptions<T>

How It Works

Rainbow DQN combines six extensions to DQN:

  1. Double Q-learning: Reduces overestimation bias
  2. Dueling networks: Separates value and advantage streams
  3. Prioritized replay: Samples important experiences more frequently
  4. Multi-step learning: Uses n-step returns for better credit assignment
  5. Distributional RL: Learns full distribution of returns (C51)
  6. Noisy networks: Parameter noise for exploration

Constructors

Constructor Summary
RainbowDQNOptions()

Properties

Property Summary
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } Gets or sets a factory for the optimizer each of this agent’s networks trains with.
List<int> AdvantageStreamLayers { get; set; } Gets or sets the widths of the advantage stream’s hidden ReLU layers, between the shared trunk and A(s, a).
List<int> SharedLayers { get; set; } Hidden layer sizes of the shared trunk that feeds both streams.
List<int> ValueStreamLayers { get; set; } Gets or sets the widths of the value stream’s hidden ReLU layers, between the shared trunk and V(s).
bool UseDistributional { get; set; } Whether to learn the full return distribution (C51) instead of its mean.
bool UseNoisyNetworks { get; set; } Whether to explore with noisy linear layers instead of epsilon-greedy.
double NoisyNetSigma { get; set; } Initial noise scale sigma_0 of the factorised noisy layer (Fortunato et al.
double PriorityAlpha { get; set; } Prioritization exponent of the replay buffer: how strongly TD error shapes sampling (0 samples uniformly).
double PriorityBeta { get; set; } Initial importance-sampling exponent that corrects the bias of prioritized replay.
double PriorityBetaIncrement { get; set; } Amount added to PriorityBeta after each sample, annealing it toward 1.
double PriorityEpsilon { get; set; } Small constant added to each priority so no transition has zero probability of being replayed.
double VMax { get; set; } Largest return represented by the distributional support.
double VMin { get; set; } Smallest return represented by the distributional support.
int ActionSize { get; set; } Number of discrete actions, or the dimension of the action vector for a continuous action space.
int NSteps { get; set; } Number of rewards (n) accumulated in the multi-step return before bootstrapping.
int NumAtoms { get; set; } Number of atoms of the categorical return distribution (51 in C51 and Rainbow).
int StateSize { get; set; } Number of features in one observation of the environment state.