A2CAgent<T>
Models & Types · AiDotNet.ReinforcementLearning.Agents.A2C
Advantage Actor-Critic (A2C) agent for reinforcement learning.
Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → A2CAgent<T>
Implements: IGradientComputable<T, Vector<T>, Vector<T>>
For Beginners
A2C learns two networks simultaneously:
- Actor: Decides which action to take (policy)
- Critic: Evaluates how good the current state is (value function)
The critic helps the actor learn faster by providing better feedback than rewards alone. Think of it like having a coach (critic) give you targeted advice instead of just saying “good” or “bad” after the game ends.
A2C is simpler than PPO but still very effective. Good starting point for actor-critic methods.
How It Works
A2C is a synchronous, simpler version of A3C that combines policy gradients with value function learning. It’s the foundation for many modern RL algorithms including PPO.
Reference: Mnih et al., “Asynchronous Methods for Deep Reinforcement Learning”, 2016 (describes A3C, A2C is the synchronous version).
Example
// Create an Advantage Actor-Critic agent for discrete action spaces
var options = new A2COptions<double> { StateSize = 4, ActionSize = 2, PolicyLearningRate = 0.001 };
var agent = new A2CAgent<double>(options);
// Select an action given the current environment state
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);
Constructors
| Constructor | Summary |
|---|---|
A2CAgent() |
Initializes a new instance with default settings. |
A2CAgent(A2COptions<T> options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
ModelOptions GetOptions() |
|
T Train() |
|
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction) |
|
Vector<T> SelectAction(Vector<T> state, bool training) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void ApplyGradients(Vector<T> gradients, T learningRate) |
|
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void LoadModel(string filepath) |
|
void SaveModel(string filepath) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool terminated, bool truncated) |