RainbowDQNOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Rainbow DQN agent.
Inheritance: ModelOptions → ReinforcementLearningOptions<T> → RainbowDQNOptions<T>
How It Works
Rainbow DQN combines six extensions to DQN:
- Double Q-learning: Reduces overestimation bias
- Dueling networks: Separates value and advantage streams
- Prioritized replay: Samples important experiences more frequently
- Multi-step learning: Uses n-step returns for better credit assignment
- Distributional RL: Learns full distribution of returns (C51)
- Noisy networks: Parameter noise for exploration
Constructors
| Constructor | Summary |
|---|---|
RainbowDQNOptions() |
Properties
| Property | Summary |
|---|---|
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } |
Gets or sets a factory for the optimizer each of this agent’s networks trains with. |
List<int> AdvantageStreamLayers { get; set; } |
Gets or sets the widths of the advantage stream’s hidden ReLU layers, between the shared trunk and A(s, a). |
List<int> SharedLayers { get; set; } |
Hidden layer sizes of the shared trunk that feeds both streams. |
List<int> ValueStreamLayers { get; set; } |
Gets or sets the widths of the value stream’s hidden ReLU layers, between the shared trunk and V(s). |
bool UseDistributional { get; set; } |
Whether to learn the full return distribution (C51) instead of its mean. |
bool UseNoisyNetworks { get; set; } |
Whether to explore with noisy linear layers instead of epsilon-greedy. |
double NoisyNetSigma { get; set; } |
Initial noise scale sigma_0 of the factorised noisy layer (Fortunato et al. |
double PriorityAlpha { get; set; } |
Prioritization exponent of the replay buffer: how strongly TD error shapes sampling (0 samples uniformly). |
double PriorityBeta { get; set; } |
Initial importance-sampling exponent that corrects the bias of prioritized replay. |
double PriorityBetaIncrement { get; set; } |
Amount added to PriorityBeta after each sample, annealing it toward 1. |
double PriorityEpsilon { get; set; } |
Small constant added to each priority so no transition has zero probability of being replayed. |
double VMax { get; set; } |
Largest return represented by the distributional support. |
double VMin { get; set; } |
Smallest return represented by the distributional support. |
int ActionSize { get; set; } |
Number of discrete actions, or the dimension of the action vector for a continuous action space. |
int NSteps { get; set; } |
Number of rewards (n) accumulated in the multi-step return before bootstrapping. |
int NumAtoms { get; set; } |
Number of atoms of the categorical return distribution (51 in C51 and Rainbow). |
int StateSize { get; set; } |
Number of features in one observation of the environment state. |