OffPolicyMonteCarloAgent<T>
Models & Types ยท AiDotNet.ReinforcementLearning.Agents.MonteCarlo
Off-Policy Monte Carlo Control agent with weighted importance sampling.
Inheritance: ReinforcementLearningAgentBase<T> โ OffPolicyMonteCarloAgent<T>
For Beginners
Off-policy MC learns the best strategy by watching someone else play (the behavior policy), then correcting for the difference using importance sampling weights. This is like learning optimal chess strategy by studying games played by beginners, adjusting for their suboptimal moves. The advantage is you can reuse old data collected under any policy, but importance sampling can introduce high variance.
How It Works
Off-Policy MC uses importance sampling to learn an optimal policy (target) while following a different exploratory policy (behavior).
Example
// Create an off-policy Monte Carlo agent with importance sampling
var options = new OffPolicyMonteCarloOptions<double> { StateSize = 4, ActionSize = 2 };
var agent = new OffPolicyMonteCarloAgent<double>(options);
// Select an action using the behavior policy
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);
Constructors
| Constructor | Summary |
|---|---|
OffPolicyMonteCarloAgent() |
Initializes a new instance with default settings. |
OffPolicyMonteCarloAgent(OffPolicyMonteCarloOptions<T> options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
T Train() |
|
Task TrainAsync() |
|
Task<Vector<T>> PredictAsync(Vector<T> input) |
|
Vector<T> Predict(Vector<T> input) |
|
Vector<T> SelectAction(Vector<T> state, bool training) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void LoadModel(string filepath) |
|
void ResetEpisode() |
|
void SaveModel(string filepath) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |