DuelingDQNAgent<T>

Models & Types · AiDotNet.ReinforcementLearning.Agents.DuelingDQN

Dueling Deep Q-Network agent for reinforcement learning.

Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → DuelingDQNAgent<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

Dueling DQN splits Q-values into two parts:

  • Value V(s): How good is this state overall?
  • Advantage A(s,a): How much better is action ‘a’ compared to average?
  • Q(s,a) = V(s) + (A(s,a) - mean(A(s,:)))

This is powerful because:

  • The agent learns state values even when actions don’t matter much
  • Faster learning in scenarios where action choice rarely matters
  • Better generalization across similar states

Example: In a car driving game, being on the road is valuable regardless of whether you accelerate slightly or not. Dueling DQN learns “being on road = good” separately from “how much to accelerate”.

How It Works

Dueling DQN separates the estimation of state value V(s) and action advantages A(s,a), allowing the network to learn which states are valuable without having to learn the effect of each action for each state. This architecture is particularly effective when many actions do not affect the state in a relevant way.

Reference: Wang et al., “Dueling Network Architectures for Deep RL”, 2016.

Example

// Create a Dueling DQN agent that separates state value from action advantages
var options = new DuelingDQNOptions<double> { StateSize = 4, ActionSize = 2, LearningRate = 0.001 };
var agent = new DuelingDQNAgent<double>(options);
            
// Select an action for the current state
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
DuelingDQNAgent() Initializes a new instance with default settings.
DuelingDQNAgent(DuelingDQNOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
ModelOptions GetOptions()
T Train()
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void Dispose() Disposes the agent and both dueling networks it owns.
void LoadModel(string filepath)
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)