DuelingDQNAgent<T>
Models & Types · AiDotNet.ReinforcementLearning.Agents.DuelingDQN
Dueling Deep Q-Network agent for reinforcement learning.
Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → DuelingDQNAgent<T>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
Dueling DQN splits Q-values into two parts:
- Value V(s): How good is this state overall?
- Advantage A(s,a): How much better is action ‘a’ compared to average?
- Q(s,a) = V(s) + (A(s,a) - mean(A(s,:)))
This is powerful because:
- The agent learns state values even when actions don’t matter much
- Faster learning in scenarios where action choice rarely matters
- Better generalization across similar states
Example: In a car driving game, being on the road is valuable regardless of whether you accelerate slightly or not. Dueling DQN learns “being on road = good” separately from “how much to accelerate”.
How It Works
Dueling DQN separates the estimation of state value V(s) and action advantages A(s,a), allowing the network to learn which states are valuable without having to learn the effect of each action for each state. This architecture is particularly effective when many actions do not affect the state in a relevant way.
Reference: Wang et al., “Dueling Network Architectures for Deep RL”, 2016.
Example
// Create a Dueling DQN agent that separates state value from action advantages
var options = new DuelingDQNOptions<double> { StateSize = 4, ActionSize = 2, LearningRate = 0.001 };
var agent = new DuelingDQNAgent<double>(options);
// Select an action for the current state
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);
Constructors
| Constructor | Summary |
|---|---|
DuelingDQNAgent() |
Initializes a new instance with default settings. |
DuelingDQNAgent(DuelingDQNOptions<T> options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
ModelOptions GetOptions() |
|
T Train() |
|
Vector<T> SelectAction(Vector<T> state, bool training) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void Dispose() |
Disposes the agent and both dueling networks it owns. |
void LoadModel(string filepath) |
|
void SaveModel(string filepath) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |