WorldModelsOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for World Models agents (Ha & Schmidhuber 2018, “Recurrent World Models Facilitate Policy Evolution”, arXiv 1803.10122).
Inheritance: ModelOptions → ReinforcementLearningOptions<T> → WorldModelsOptions<T>
For Beginners
World Models is inspired by how humans learn: we build mental models of the world, then make decisions based on those models rather than raw sensory input.
Key components:
- VAE (V): Compresses visual observations into compact latent codes
- MDN-RNN (M): Learns temporal dynamics (what happens next)
- Controller (C): A tiny policy acting on V’s code and M’s memory
- Learning in Dreams: The controller is evolved inside rollouts imagined by M
Think of it like: First, learn to compress images (VAE). Then, learn how compressed images change over time (RNN). Finally, learn to act based on compressed predictions (controller).
How It Works
World Models has three parts. V is a variational autoencoder that compresses each observation into a latent code z. M is a mixture-density recurrent network (an LSTM whose output parameterises a Gaussian mixture over the next z) that models how z evolves under an action. C is a small controller mapping [z, h] to an action, trained with CMA-ES. The defaults below are the paper’s Car Racing configuration unless a property says otherwise.
Constructors
| Constructor | Summary |
|---|---|
WorldModelsOptions() |
Properties
| Property | Summary |
|---|---|
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } |
Gets or sets a factory for the optimizer each of this agent’s networks trains with. |
List<int> ControllerLayers { get; set; } |
Gets the widths of the controller’s hidden tanh layers. |
List<int> VAEEncoderChannels { get; set; } |
Gets the widths of V’s hidden encoder layers (mirrored, reversed, in the decoder). |
double ControllerInitialStepSize { get; set; } |
Gets the initial CMA-ES step size (the standard deviation of the first generation around the current controller). |
double Temperature { get; set; } |
Gets the sampling temperature tau used when M generates the next latent in a dream. |
double VAEBeta { get; set; } |
Gets the weight of the KL term in V’s loss. |
int ActionSize { get; set; } |
Gets the dimension of the continuous action. |
int ControllerEvaluationRollouts { get; set; } |
Gets the number of dream rollouts each candidate controller’s fitness is averaged over. |
int ControllerGenerations { get; set; } |
Gets the number of CMA-ES generations the controller is evolved for in each training cycle. |
int ControllerPopulationSize { get; set; } |
Gets the number of candidate controllers CMA-ES samples per generation. |
int LatentSize { get; set; } |
Gets the size of V’s latent code z. |
int NumMixtures { get; set; } |
Gets the number of Gaussian components in M’s mixture density output. |
int ObservationChannels { get; set; } |
Gets the number of observation channels. |
int ObservationHeight { get; set; } |
Gets the observation height. |
int ObservationWidth { get; set; } |
Gets the observation width. |
int RNNEpochs { get; set; } |
Gets the number of passes over the stored sequences M trains for in each training cycle. |
int RNNHiddenSize { get; set; } |
Gets the number of hidden units in each of M’s LSTM layers. |
int RNNLayers { get; set; } |
Gets the number of stacked LSTM layers in M. |
int RNNSequenceLength { get; set; } |
Gets the maximum length of the sequences M is trained on. |
int RolloutLength { get; set; } |
Gets the maximum number of steps of each dream rollout used to score a controller. |
int VAEEpochs { get; set; } |
Gets the number of passes over the stored observations V trains for in each training cycle. |