DynaQPlusAgent<T>

Models & Types ยท AiDotNet.ReinforcementLearning.Agents.Planning

Dyna-Q+ agent with exploration bonus for handling changing environments.

Inheritance: ReinforcementLearningAgentBase<T> โ†’ DynaQPlusAgent<T>

For Beginners

Dyna-Q+ extends Dyna-Q with an exploration bonus that encourages revisiting states not seen recently. This is crucial in changing environments where the optimal strategy may shift over time. The bonus grows with time since last visit, ensuring the agent periodically re-explores to detect environmental changes. Think of it as a curious learner who checks old paths to see if anything changed.

Example

// Create a Dyna-Q+ agent with exploration bonus for changing environments
var options = new DynaQPlusOptions<double> { PlanningSteps = 10, Kappa = 0.001, StateSize = 4, ActionSize = 2 };
var agent = new DynaQPlusAgent<double>(options);
            
// Select an action with bonus for revisiting under-explored states
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
DynaQPlusAgent() Initializes a new instance with default settings.
DynaQPlusAgent(DynaQPlusOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void ResetEpisode()
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)