OnPolicyMonteCarloAgent<T>

Models & Types ยท AiDotNet.ReinforcementLearning.Agents.MonteCarlo

On-Policy Monte Carlo Control agent with epsilon-greedy exploration.

Inheritance: ReinforcementLearningAgentBase<T> โ†’ OnPolicyMonteCarloAgent<T>

For Beginners

On-policy MC learns by evaluating the same policy it uses to collect data. The agent follows an epsilon-greedy strategy (mostly best action, sometimes random) and improves that exact strategy over time. Think of learning to cook by actually cooking with your current recipe, then adjusting based on results. Simpler than off-policy methods but cannot reuse data from previous policies.

How It Works

On-Policy MC Control uses epsilon-greedy policy for both behavior and target, ensuring exploration while learning the optimal policy.

Example

// Create an on-policy Monte Carlo agent with epsilon-greedy exploration
var options = new OnPolicyMonteCarloOptions<double> { StateSize = 4, ActionSize = 2 };
var agent = new OnPolicyMonteCarloAgent<double>(options);
            
// Select an action following the epsilon-greedy policy
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
OnPolicyMonteCarloAgent() Initializes a new instance with default settings.
OnPolicyMonteCarloAgent(OnPolicyMonteCarloOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void ResetEpisode()
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)