QMIXOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for QMIX agents (Rashid et al. 2018).

Inheritance: ModelOptions → ReinforcementLearningOptions<T> → QMIXOptions<T>

For Beginners

QMIX learns a team of agents that each choose their own action from what they see, while training them together: a mixing network turns the agents’ values into one team value that can only rise when any agent’s value rises, so each agent picking its own best action also picks the team’s best joint action.

How It Works

The defaults are the paper’s (§5 and Appendix C): a shared GRU agent network of 64 units, a mixing network of 32 units whose weights single-layer hypernetworks generate, RMSprop at 5e-4, batches of 32 episodes from a buffer of the 5000 most recent, the target networks updated every 200 episodes, ε annealed linearly from 1 to 0.05 over 50000 steps, γ = 0.99, and the summed squared TD error of eq. 6 against the target network’s max. Where the paper is silent, pymarl (the authors’ code) fills in: RMSprop’s α = 0.99 and ε = 1e-5 and the gradient norm clipped at 10. pymarl’s own configuration differs in three places, each an option: HypernetLayers = 2, DoubleQ and MeanLoss.

Constructors

Constructor Summary
QMIXOptions() Creates the options with the paper’s defaults.

Properties

Property Summary
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } Creates the optimizer for the joint agent-and-mixer update.
IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> Optimizer { get; set; } Replaces the paper’s RMSprop.
bool DoubleQ { get; set; } Double Q-learning targets: the target networks evaluate the live agent network’s greedy action rather than their own max.
bool IncludeAgentId { get; set; } Whether each agent’s input includes its id, one-hot, so the shared network can tell agents apart (yes).
bool IncludeLastAction { get; set; } Whether each agent’s input includes its previous action, one-hot (the paper and pymarl: yes).
bool MeanLoss { get; set; } Averages the squared TD error over the batch’s filled steps rather than summing it (the paper’s eq.
double RmsPropAlpha { get; set; } RMSprop’s smoothing constant α (pymarl: 0.99).
double RmsPropEpsilon { get; set; } RMSprop’s ε (pymarl: 1e-5).
int ActionSize { get; set; } The number of discrete actions each agent chooses from.
int EpsilonAnnealSteps { get; set; } The environment steps over which ε anneals linearly from EpsilonStart to EpsilonEnd (50000).
int GlobalStateSize { get; set; } The size of the global state the mixing network’s hypernetworks read.
int HypernetEmbed { get; set; } The hidden width of two-layer hypernetworks (pymarl: 64).
int HypernetLayers { get; set; } The layers in each weight hypernetwork: 1 in the paper (a linear layer), 2 in pymarl’s qmix.yaml (a ReLU layer of HypernetEmbed units, then the linear one).
int MixingEmbedDim { get; set; } The mixing network’s hidden width (the paper and pymarl: 32).
int NumAgents { get; set; } The number of agents.
int RnnHiddenDim { get; set; } The GRU’s width in the agent network (the paper and pymarl: 64).
int StateSize { get; set; } The size of each agent’s own observation.