DuelingCombinationLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Dueling DQN combination head (Wang et al. 2016, “Dueling Network Architectures for Deep Reinforcement Learning”). Splits the shared feature trunk’s output into a scalar state-value V(s) and an action-advantage vector A(s, a), then combines them as Q(s, a) = V(s) + (A(s, a) − mean_a A(s, a)) to enforce the identifiability constraint Eq. 9 of the paper.
Inheritance: LayerBase<T> → DuelingCombinationLayer<T>
Implements: IShapeContract
For Beginners
“Dueling” means the network learns the value of being in a state separately from the relative benefit of each action. V(s) answers “is this state good?”, A(s, a) answers “given this state, is action a better or worse than average?”. Combining them gives more stable training than predicting Q(s, a) directly with one big head — the value head doesn’t have to redundantly encode “this state is good” for every action.
How It Works
What this layer does: Reads [batch, featureDim] shared
trunk features and emits [batch, actionSize] Q-values. Internally
holds two parallel linear projections — a single-unit value head and an
actionSize-unit advantage head — and combines them using the
mean-subtraction form (paper §3, Eq. 9). The mean form is what every
production Rainbow / Double-DQN implementation uses (e.g.
Stable-Baselines3 DuelingQ, RLlib DuelingMixin) because the
max-subtraction alternative is unstable under bootstrapping.
All forward arithmetic runs through Engine
ops on the trainable tensor instances, so the gradient tape captures
gradients for both heads automatically.
Constructors
| Constructor | Summary |
|---|---|
DuelingCombinationLayer(int featureDim, int actionSize, int numAtoms, int[] valueHiddenWidths, int[] advantageHiddenWidths, int? seed) |
Initializes a dueling head: by default one Q-value per action with linear value and advantage heads, and optionally hidden stream layers and, for a distributional agent, one value per atom. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, string> GetParameterRoles() |
Returns parameter roles for per-role optimizer configuration (e.g., weight decay exemption for biases). |
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
IReadOnlyList<Tensor<T>> GetTrainableParameters() |
Returns all trainable parameter tensors marked with [TrainableParameter]. |
void ResetState() |
|
void SetTrainableParameters(IReadOnlyList<Tensor<T>> parameters) |
Replaces trainable parameter tensors (e.g., with ParameterBuffer views). |
void UpdateParameters(T learningRate) |
|
void ZeroGrad() |
Clears all gradient fields discovered by convention ({paramName}Gradient). |