A3COptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for Asynchronous Advantage Actor-Critic (A3C) agents.

Inheritance: ModelOptions → ReinforcementLearningOptions<T> → A3COptions<T>

For Beginners

A3C is like having multiple students learn the same subject simultaneously, each with different experiences. They periodically share what they learned with a central “teacher” (global network), and everyone benefits from the combined knowledge.

Key features:

  • Asynchronous: Multiple agents run in parallel
  • Actor-Critic: Learns both policy and value function
  • No Replay Buffer: Uses on-policy learning
  • Diverse Exploration: Different agents explore different strategies

Famous for: DeepMind’s breakthrough paper (2016), enables CPU-only training

How It Works

A3C runs multiple agents in parallel, each learning from different experiences. The parallel exploration provides diverse training data and stabilizes learning.

Constructors

Constructor Summary
A3COptions() Creates options with the default values.

Properties

Property Summary
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } Gets or sets a factory for the optimizer each of this agent’s networks trains with.
ILossFunction<T> ValueLossFunction { get; set; } Loss function used to train the value network.
List<int> PolicyHiddenLayers { get; set; } Hidden layer sizes of the policy network.
List<int> ValueHiddenLayers { get; set; } Hidden layer sizes of the value network.
T EntropyCoefficient { get; set; } Weight of the entropy bonus in the objective, which encourages exploration.
T PolicyLearningRate { get; set; } Learning rate of the policy network.
T ValueLearningRate { get; set; } Learning rate of the value network.
bool IsContinuous { get; set; } Whether the action space is continuous rather than discrete.
int ActionSize { get; set; } Number of discrete actions, or the dimension of the action vector for a continuous action space.
int NumWorkers { get; set; } Number of parallel worker agents that explore and update the shared global network.
int StateSize { get; set; } Number of features in one observation of the environment state.
int TMax { get; set; } Maximum number of steps a worker takes before it pushes its gradients to the global network (t_max in Mnih et al.