REINFORCEAgent<T>
Models & Types · AiDotNet.ReinforcementLearning.Agents.REINFORCE
REINFORCE (Monte Carlo Policy Gradient) agent for reinforcement learning.
Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → REINFORCEAgent<T>
Implements: IGradientComputable<T, Vector<T>, Vector<T>>
For Beginners
REINFORCE is the “hello world” of policy gradient methods. The algorithm is beautifully simple:
- Play an entire episode
- Calculate total rewards for each action
- Make good actions more likely, bad actions less likely
Think of it like learning to play a game:
- You play a round
- At the end, you see your score
- You adjust your strategy to do better next time
Pros: Simple, works for any problem, easy to understand Cons: High variance, slow learning, requires complete episodes
Modern algorithms like PPO and A2C improve on REINFORCE’s core ideas.
How It Works
REINFORCE is the simplest and most fundamental policy gradient algorithm. It directly optimizes the policy by following the gradient of expected returns. Despite its simplicity, it forms the foundation for many modern RL algorithms.
Reference: Williams, R. J. (1992). “Simple statistical gradient-following algorithms for connectionist RL.”
Example
// Create a REINFORCE agent for Monte Carlo policy gradient learning
var options = new REINFORCEOptions<double> { StateSize = 4, ActionSize = 2, LearningRate = 0.001 };
var agent = new REINFORCEAgent<double>(options);
// Select an action sampled from the learned policy
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);
Constructors
| Constructor | Summary |
|---|---|
REINFORCEAgent() |
|
REINFORCEAgent(REINFORCEOptions<T> options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
ModelOptions GetOptions() |
|
T Train() |
|
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction) |
|
Vector<T> SelectAction(Vector<T> state, bool training) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void ApplyGradients(Vector<T> gradients, T learningRate) |
|
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void LoadModel(string filepath) |
|
void SaveModel(string filepath) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |