SARSALambdaAgent<T>
Models & Types ยท AiDotNet.ReinforcementLearning.Agents.EligibilityTraces
SARSA(lambda) agent that combines on-policy SARSA control with eligibility traces for faster credit assignment while respecting the current exploration policy.
Inheritance: ReinforcementLearningAgentBase<T> โ SARSALambdaAgent<T>
For Beginners
SARSA(lambda) combines two ideas:
- SARSA: Learn from the actions you actually take (on-policy, safer)
- Eligibility traces: Remember and update recent states when rewards arrive
This means the agent learns faster than plain SARSA (credit propagates back through the trajectory) while still being cautious about risky actions.
The lambda parameter controls the trace decay:
- lambda = 0: Same as regular SARSA (1-step updates)
- lambda = 1: Full Monte Carlo returns (high variance)
- lambda = 0.9: Good default balance
Best for: Environments where safety matters AND episodes are long.
How It Works
SARSA(lambda) extends SARSA by maintaining eligibility traces that propagate temporal-difference errors backward through recently visited state-action pairs. Unlike Q(lambda), SARSA(lambda) is on-policy: it updates based on the action actually taken, making it safer in environments with dangerous states.
Example
// Create an on-policy SARSA(lambda) agent with eligibility traces
var options = new SARSALambdaOptions<double> { Lambda = 0.9, LearningRate = 0.1, StateSize = 4, ActionSize = 2 };
var agent = new SARSALambdaAgent<double>(options);
// Select an action for the current state
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);
Constructors
| Constructor | Summary |
|---|---|
SARSALambdaAgent() |
Initializes a new instance with default settings. |
SARSALambdaAgent(SARSALambdaOptions<T> options) |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> GetMetrics() |
|
ModelMetadata<T> GetModelMetadata() |
|
T Train() |
|
Task TrainAsync() |
|
Task<Vector<T>> PredictAsync(Vector<T> input) |
|
Vector<T> Predict(Vector<T> input) |
|
Vector<T> SelectAction(Vector<T> state, bool training) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void LoadModel(string filepath) |
|
void ResetEpisode() |
|
void SaveModel(string filepath) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |