MuZeroAgent<T>

Models & Types · AiDotNet.ReinforcementLearning.Agents.MuZero

MuZero agent combining tree search with learned models.

Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → MuZeroAgent<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

MuZero is DeepMind’s breakthrough that achieved superhuman performance in Atari, Go, Chess, and Shogi without being told the rules. It learns its own “internal model” of the game and uses tree search to plan ahead.

Three key networks:

  • Representation: Observation -> hidden state
  • Dynamics: (hidden state, action) -> (next hidden state, reward)
  • Prediction: hidden state -> (policy, value)

Plus tree search (MCTS) for planning using the learned model.

Think of it as: Learning chess by watching games, figuring out the rules yourself, then planning moves by mentally simulating the game.

Famous for: Superhuman Atari/board games without knowing rules

How It Works

MuZero combines tree search (like AlphaZero) with learned dynamics. It masters games without knowing the rules, learning its own internal model.

Example

// Create a MuZero agent that learns its own world model
var options = new MuZeroOptions<double> { LatentStateSize = 64, ActionSize = 4, NumSimulations = 50 };
var agent = new MuZeroAgent<double>(options);
            
// Select an action using Monte Carlo Tree Search with learned dynamics
var state = new Vector<double>(new double[64]);
var action = agent.SelectAction(state);

Constructors

Constructor Summary
MuZeroAgent() Initializes a new instance with default settings.
MuZeroAgent(MuZeroOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input) IFullModel.Predict surfaces the raw prediction-network output (policy logits + value) for the input observation rather than the one-hot committed action.
Vector<T> SelectAction(Vector<T> observation, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void ResetEpisode()
void SaveModel(string filepath)
void StoreExperience(Vector<T> observation, Vector<T> action, T reward, Vector<T> nextObservation, bool done)
void Train(Vector<T> state, Vector<T> target) Supervised one-shot training from a (state, target) pair — the entry point the model-family invariant harness and any non-RL caller use.