QMIXOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for QMIX agents (Rashid et al. 2018).
Inheritance: ModelOptions → ReinforcementLearningOptions<T> → QMIXOptions<T>
For Beginners
QMIX learns a team of agents that each choose their own action from what they see, while training them together: a mixing network turns the agents’ values into one team value that can only rise when any agent’s value rises, so each agent picking its own best action also picks the team’s best joint action.
How It Works
The defaults are the paper’s (§5 and Appendix C): a shared GRU agent network of 64 units, a mixing network of 32
units whose weights single-layer hypernetworks generate, RMSprop at 5e-4, batches of 32 episodes from a buffer of
the 5000 most recent, the target networks updated every 200 episodes, ε annealed linearly from 1 to 0.05 over
50000 steps, γ = 0.99, and the summed squared TD error of eq. 6 against the target network’s max. Where the paper is
silent, pymarl (the authors’ code) fills in: RMSprop’s α = 0.99 and ε = 1e-5 and the gradient norm clipped at 10.
pymarl’s own configuration differs in three places, each an option: HypernetLayers = 2,
DoubleQ and MeanLoss.
Constructors
| Constructor | Summary |
|---|---|
QMIXOptions() |
Creates the options with the paper’s defaults. |
Properties
| Property | Summary |
|---|---|
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } |
Creates the optimizer for the joint agent-and-mixer update. |
IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> Optimizer { get; set; } |
Replaces the paper’s RMSprop. |
bool DoubleQ { get; set; } |
Double Q-learning targets: the target networks evaluate the live agent network’s greedy action rather than their own max. |
bool IncludeAgentId { get; set; } |
Whether each agent’s input includes its id, one-hot, so the shared network can tell agents apart (yes). |
bool IncludeLastAction { get; set; } |
Whether each agent’s input includes its previous action, one-hot (the paper and pymarl: yes). |
bool MeanLoss { get; set; } |
Averages the squared TD error over the batch’s filled steps rather than summing it (the paper’s eq. |
double RmsPropAlpha { get; set; } |
RMSprop’s smoothing constant α (pymarl: 0.99). |
double RmsPropEpsilon { get; set; } |
RMSprop’s ε (pymarl: 1e-5). |
int ActionSize { get; set; } |
The number of discrete actions each agent chooses from. |
int EpsilonAnnealSteps { get; set; } |
The environment steps over which ε anneals linearly from EpsilonStart to EpsilonEnd (50000). |
int GlobalStateSize { get; set; } |
The size of the global state the mixing network’s hypernetworks read. |
int HypernetEmbed { get; set; } |
The hidden width of two-layer hypernetworks (pymarl: 64). |
int HypernetLayers { get; set; } |
The layers in each weight hypernetwork: 1 in the paper (a linear layer), 2 in pymarl’s qmix.yaml (a ReLU layer of HypernetEmbed units, then the linear one). |
int MixingEmbedDim { get; set; } |
The mixing network’s hidden width (the paper and pymarl: 32). |
int NumAgents { get; set; } |
The number of agents. |
int RnnHiddenDim { get; set; } |
The GRU’s width in the agent network (the paper and pymarl: 64). |
int StateSize { get; set; } |
The size of each agent’s own observation. |