ThompsonSamplingExploration<T>

Models & Types ยท AiDotNet.ReinforcementLearning.Policies.Exploration

Thompson Sampling (Bayesian) exploration for discrete action spaces. Maintains Beta distributions for each action and samples from posteriors.

Inheritance: ExplorationStrategyBase<T> โ†’ ThompsonSamplingExploration<T>

Example

var random = new Random(42);
var state = new Vector<double>(new double[] { 0.1, -0.4, 0.7, 0.2 });
// Create Thompson Sampling exploration with uniform Beta priors
var exploration = new ThompsonSamplingExploration<double>(priorAlpha: 1.0, priorBeta: 1.0);
            
// Sample from posterior Beta distributions to select actions
var policyAction = new Vector<double>(new double[] { 0.0, 1.0, 0.0 });
var action = exploration.GetExplorationAction(state, policyAction, actionSpaceSize: 3, random);

Constructors

Constructor Summary
ThompsonSamplingExploration(double priorAlpha, double priorBeta) Initializes a new instance of the Thompson Sampling exploration strategy.

Methods

Method Summary
Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random) Selects action by sampling from Beta posteriors for each action.
void Reset() Resets all action distributions to prior.
void Update() Updates internal parameters (call UpdateDistribution separately for each action).
void UpdateDistribution(int actionIndex, double reward) Updates the Beta distribution for a specific action based on reward.