REINFORCEAgent<T>

Models & Types · AiDotNet.ReinforcementLearning.Agents.REINFORCE

REINFORCE (Monte Carlo Policy Gradient) agent for reinforcement learning.

Inheritance: ReinforcementLearningAgentBase<T> → DeepReinforcementLearningAgentBase<T> → REINFORCEAgent<T>

Implements: IGradientComputable<T, Vector<T>, Vector<T>>

For Beginners

REINFORCE is the “hello world” of policy gradient methods. The algorithm is beautifully simple:

  1. Play an entire episode
  2. Calculate total rewards for each action
  3. Make good actions more likely, bad actions less likely

Think of it like learning to play a game:

  • You play a round
  • At the end, you see your score
  • You adjust your strategy to do better next time

Pros: Simple, works for any problem, easy to understand Cons: High variance, slow learning, requires complete episodes

Modern algorithms like PPO and A2C improve on REINFORCE’s core ideas.

How It Works

REINFORCE is the simplest and most fundamental policy gradient algorithm. It directly optimizes the policy by following the gradient of expected returns. Despite its simplicity, it forms the foundation for many modern RL algorithms.

Reference: Williams, R. J. (1992). “Simple statistical gradient-following algorithms for connectionist RL.”

Example

// Create a REINFORCE agent for Monte Carlo policy gradient learning
var options = new REINFORCEOptions<double> { StateSize = 4, ActionSize = 2, LearningRate = 0.001 };
var agent = new REINFORCEAgent<double>(options);
            
// Select an action sampled from the learned policy
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
REINFORCEAgent()
REINFORCEAgent(REINFORCEOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
ModelOptions GetOptions()
T Train()
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void ApplyGradients(Vector<T> gradients, T learningRate)
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)