TabularQLearningAgent<T>

Models & Types · AiDotNet.ReinforcementLearning.Agents.TabularQLearning

Tabular Q-Learning agent using lookup table for Q-values.

Inheritance: ReinforcementLearningAgentBase<T> → TabularQLearningAgent<T>

Implements: IGradientComputable<T, Vector<T>, Vector<T>>

For Beginners

Q-Learning is like creating a cheat sheet: for every situation (state) and action you could take, you write down how good that choice is (Q-value). Over time, you update this sheet based on actual rewards you receive.

Key features:

  • Off-Policy: Learns optimal policy while following exploratory policy
  • Tabular: Uses lookup table, no function approximation
  • Model-Free: Doesn’t need to know environment dynamics
  • Value-Based: Learns action values, derives policy from them

Perfect for: Small discrete state/action spaces (grid worlds, simple games) Famous for: Watkins 1989, the foundation of modern RL

How It Works

Tabular Q-Learning is the foundational RL algorithm that maintains a table of Q-values for each state-action pair. No neural networks required.

Example

// Create a tabular Q-Learning agent for discrete state-action spaces
var options = new TabularQLearningOptions<double> { LearningRate = 0.1, StateSize = 4, ActionSize = 2 };
var agent = new TabularQLearningAgent<double>(options);
            
// Select an action using epsilon-greedy exploration
var state = new Vector<double>(new double[] { 0.5, -0.3, 1.0, 0.2 });
var action = agent.SelectAction(state);

Constructors

Constructor Summary
TabularQLearningAgent() Initializes a new instance with default settings.
TabularQLearningAgent(TabularQLearningOptions<T> options)

Properties

Property Summary
int FeatureCount { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
T Train()
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction)
Vector<T> SelectAction(Vector<T> state, bool training)
byte[] Serialize() Auto-generated common model serialization surface.
void ApplyGradients(Vector<T> gradients, T learningRate)
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath)
void SaveModel(string filepath)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)