MuZeroOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for MuZero agents.

Inheritance: ModelOptions → ReinforcementLearningOptions<T> → MuZeroOptions<T>

For Beginners

MuZero is DeepMind’s breakthrough that mastered Atari, Go, Chess, and Shogi without being told the rules. It learns its own “internal model” of the game and uses tree search to plan ahead.

Key innovations:

  • Learned Model: No need for game rules, learns environment dynamics
  • MCTS: Uses Monte Carlo Tree Search for planning
  • Three Networks: Representation, dynamics, and prediction
  • Planning: Searches through imagined futures

Think of it like: Learning to play chess by watching games, figuring out the rules yourself, then planning moves by mentally simulating the game.

Famous for: Superhuman performance across Atari, board games, without rules

How It Works

MuZero combines tree search (like AlphaZero) with learned models. It learns dynamics, rewards, and values without knowing environment rules.

Constructors

Constructor Summary
MuZeroOptions() Default constructor required for object-initializer syntax.
MuZeroOptions(MuZeroOptions<T> other) Copy constructor required by the Options golden pattern so Clone() faithfully preserves every property.

Properties

Property Summary
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } Gets or sets a factory for the optimizer of the agent’s joint training step.
List<int> DynamicsLayers { get; set; } Hidden layer sizes of the dynamics network (next latent state and reward).
List<int> PredictionLayers { get; set; } Hidden layer sizes of the prediction network (policy and value).
List<int> RepresentationLayers { get; set; } Hidden layer sizes of the representation network (observation to latent state).
double PUCTConstant { get; set; } Exploration constant of the pUCT rule that balances prior and value during tree search.
double PriorityAlpha { get; set; } Prioritization exponent of the replay buffer: a window is drawn with probability proportional to p^alpha, where p = |v(o0) - z0| is the gap between the predicted root value and its n-step target (0 samples uniformly).
double RootDirichletAlpha { get; set; } Concentration of the Dirichlet noise added to the root prior.
double RootExplorationFraction { get; set; } Fraction of the root prior replaced by Dirichlet noise during search, for exploration.
double VisitTemperature { get; set; } Gets or sets the temperature T of the visit-count distribution that training-time actions are sampled from.
int ActionSize { get; set; } Number of discrete actions the agent can choose from.
int LatentStateSize { get; set; } Size of the latent state the model plans in.
int NumSimulations { get; set; } Number of Monte Carlo tree search simulations per action.
int ObservationSize { get; set; } Dimensionality of the environment’s observation vector.
int TDSteps { get; set; } Number of steps (n) in the n-step bootstrapped value target.
int UnrollSteps { get; set; } Number of steps (K) the dynamics network is unrolled during training.