GradientBanditAgent<T>

Models & Types ยท AiDotNet.ReinforcementLearning.Agents.Bandits

Gradient Bandit agent using softmax action preferences.

Inheritance: ReinforcementLearningAgentBase<T> โ†’ GradientBanditAgent<T>

For Beginners

Instead of estimating how good each action is (like epsilon-greedy), the gradient bandit learns preferences for each action using gradient ascent. Actions with higher preferences are selected more often via softmax probabilities. When an action does better than average, its preference increases; when worse, it decreases. This approach naturally handles the exploration-exploitation trade-off through the softmax distribution without needing an explicit epsilon parameter.

Example

// Create a gradient bandit agent using softmax action selection
var options = new GradientBanditOptions<double> { NumArms = 10};
var agent = new GradientBanditAgent<double>(options);
            
// Select an arm based on learned preferences
var state = new Vector<double>(new double[] { 1.0 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
GradientBanditAgent() Initializes a new instance with default settings.
GradientBanditAgent(GradientBanditOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
Dictionary<string, T> GetMetrics()
ModelMetadata<T> GetModelMetadata()
T Train()
Task TrainAsync()
Task<Vector<T>> PredictAsync(Vector<T> input)
Vector<T> Predict(Vector<T> input)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void ResetEpisode()
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)