LSTDAgent<T>

Models & Types ยท AiDotNet.ReinforcementLearning.Agents.AdvancedRL

LSTD (Least-Squares Temporal Difference) agent using direct solution for value function weights.

Inheritance: ReinforcementLearningAgentBase<T> โ†’ LSTDAgent<T>

For Beginners

LSTD computes the value function weights directly using matrix operations instead of iterative updates. While standard TD learning takes many small steps toward the answer, LSTD solves for the answer in one computation (like solving a system of equations). This is much more sample-efficient but uses more memory and compute per update. Best for problems with linear function approximation and moderate state spaces.

Example

// Create an LSTD agent that solves value function weights directly
var options = new LSTDOptions<double> { FeatureSize = 4, ActionSize = 2 };
var agent = new LSTDAgent<double>(options);
            
// Select an action using the computed value function
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
LSTDAgent() Initializes a new instance with default settings.
LSTDAgent(LSTDOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void ResetEpisode()
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)