UpperConfidenceBoundExploration<T>
Models & Types · AiDotNet.ReinforcementLearning.Policies.Exploration
Upper Confidence Bound (UCB) exploration for discrete action spaces. Balances exploration and exploitation using confidence intervals: UCB(a) = Q(a) + c * √(ln(t) / N(a))
Inheritance: ExplorationStrategyBase<T> → UpperConfidenceBoundExploration<T>
Example
var random = new Random(42);
var state = new Vector<double>(new double[] { 0.1, -0.4, 0.7, 0.2 });
// Create UCB exploration that favors under-explored actions
var exploration = new UpperConfidenceBoundExploration<double>(explorationConstant: 2.0);
// Select action using upper confidence bound: Q(a) + c * sqrt(ln(t) / N(a))
var policyAction = new Vector<double>(new double[] { 1.5, 0.8, 2.3 });
var action = exploration.GetExplorationAction(state, policyAction, actionSpaceSize: 3, random);
Constructors
| Constructor | Summary |
|---|---|
UpperConfidenceBoundExploration(double explorationConstant) |
Initializes a new instance of the Upper Confidence Bound exploration strategy. |
Properties
| Property | Summary |
|---|---|
double ExplorationConstant { get; } |
Gets the current exploration constant. |
int TotalSteps { get; } |
Gets the total number of steps taken. |
Methods
| Method | Summary |
|---|---|
Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random) |
Selects action using UCB: action with highest Q(a) + c * √(ln(t) / N(a)) |
void Reset() |
Resets action counts and total steps. |
void Update() |
Updates internal parameters (UCB is count-based, no explicit decay). |