A3COptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Asynchronous Advantage Actor-Critic (A3C) agents.
Inheritance: ModelOptions → ReinforcementLearningOptions<T> → A3COptions<T>
For Beginners
A3C is like having multiple students learn the same subject simultaneously, each with different experiences. They periodically share what they learned with a central “teacher” (global network), and everyone benefits from the combined knowledge.
Key features:
- Asynchronous: Multiple agents run in parallel
- Actor-Critic: Learns both policy and value function
- No Replay Buffer: Uses on-policy learning
- Diverse Exploration: Different agents explore different strategies
Famous for: DeepMind’s breakthrough paper (2016), enables CPU-only training
How It Works
A3C runs multiple agents in parallel, each learning from different experiences. The parallel exploration provides diverse training data and stabilizes learning.
Constructors
| Constructor | Summary |
|---|---|
A3COptions() |
Creates options with the default values. |
Properties
| Property | Summary |
|---|---|
Func<IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>>> OptimizerFactory { get; set; } |
Gets or sets a factory for the optimizer each of this agent’s networks trains with. |
ILossFunction<T> ValueLossFunction { get; set; } |
Loss function used to train the value network. |
List<int> PolicyHiddenLayers { get; set; } |
Hidden layer sizes of the policy network. |
List<int> ValueHiddenLayers { get; set; } |
Hidden layer sizes of the value network. |
T EntropyCoefficient { get; set; } |
Weight of the entropy bonus in the objective, which encourages exploration. |
T PolicyLearningRate { get; set; } |
Learning rate of the policy network. |
T ValueLearningRate { get; set; } |
Learning rate of the value network. |
bool IsContinuous { get; set; } |
Whether the action space is continuous rather than discrete. |
int ActionSize { get; set; } |
Number of discrete actions, or the dimension of the action vector for a continuous action space. |
int NumWorkers { get; set; } |
Number of parallel worker agents that explore and update the shared global network. |
int StateSize { get; set; } |
Number of features in one observation of the environment state. |
int TMax { get; set; } |
Maximum number of steps a worker takes before it pushes its gradients to the global network (t_max in Mnih et al. |