ThompsonSamplingExploration<T>
Models & Types ยท AiDotNet.ReinforcementLearning.Policies.Exploration
Thompson Sampling (Bayesian) exploration for discrete action spaces. Maintains Beta distributions for each action and samples from posteriors.
Inheritance: ExplorationStrategyBase<T> โ ThompsonSamplingExploration<T>
Example
var random = new Random(42);
var state = new Vector<double>(new double[] { 0.1, -0.4, 0.7, 0.2 });
// Create Thompson Sampling exploration with uniform Beta priors
var exploration = new ThompsonSamplingExploration<double>(priorAlpha: 1.0, priorBeta: 1.0);
// Sample from posterior Beta distributions to select actions
var policyAction = new Vector<double>(new double[] { 0.0, 1.0, 0.0 });
var action = exploration.GetExplorationAction(state, policyAction, actionSpaceSize: 3, random);
Constructors
| Constructor | Summary |
|---|---|
ThompsonSamplingExploration(double priorAlpha, double priorBeta) |
Initializes a new instance of the Thompson Sampling exploration strategy. |
Methods
| Method | Summary |
|---|---|
Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random) |
Selects action by sampling from Beta posteriors for each action. |
void Reset() |
Resets all action distributions to prior. |
void Update() |
Updates internal parameters (call UpdateDistribution separately for each action). |
void UpdateDistribution(int actionIndex, double reward) |
Updates the Beta distribution for a specific action based on reward. |