Experience<T, TState, TAction>
Models & Types · AiDotNet.ReinforcementLearning.ReplayBuffers
Represents a single experience tuple (s, a, r, s’, done) for reinforcement learning.
Implements: IEquatable<Experience<T, TState, TAction>>
Known derived types: Experience<T>
For Beginners
An experience is one step of interaction with the environment. It contains everything the agent needs to learn from that step:
- State: What the situation looked like before the agent acted (like a snapshot)
- Action: What the agent decided to do
- Reward: The feedback received (positive = good, negative = bad, zero = neutral)
- NextState: What the situation looks like after the action
- Done: Whether this action ended the episode (game over, goal reached, etc.)
For example, in a maze-solving robot:
- State: Robot’s current position and sensor readings
- Action: “move forward” or “turn left”
- Reward: +10 for reaching the exit, -1 for hitting a wall, 0 otherwise
- NextState: Robot’s new position after the action
- Done: True if robot reached the exit or got stuck
Common Type Combinations:
Experience<double, Vector<double>, Vector<double>>- For continuous actions (e.g., robotic control)Experience<double, Vector<double>, int>- For discrete actions (e.g., game playing)Experience<float, Tensor<float>, int>- For image-based states (e.g., Atari games)
How It Works
An Experience is a fundamental data structure in reinforcement learning that captures a single interaction between an agent and its environment. It consists of five components: the current state, the action taken, the reward received, the resulting next state, and a flag indicating whether the episode has ended. This tuple is used to train reinforcement learning agents in algorithms like Q-learning, Deep Q-Networks (DQN), PPO, and many others.
Constructors
| Constructor | Summary |
|---|---|
Experience(TState State, TAction Action, T Reward, TState NextState, bool Done) |
Represents a single experience tuple (s, a, r, s’, done) for reinforcement learning. |
Properties
| Property | Summary |
|---|---|
T Reward { get; set; } |
|
TAction Action { get; set; } |
|
TState NextState { get; set; } |
|
TState State { get; set; } |
|
bool Done { get; set; } |
|
bool[] NextLegalActions { get; set; } |
Snapshot of legal next-state actions; null means unrestricted or terminal. |
double Priority { get; set; } |
Gets or sets the priority for prioritized experience replay. |
Methods
| Method | Summary |
|---|---|
Experience<T, TState, TAction> <Clone>$() |
|
bool Equals(Experience<T, TState, TAction> other) |
|
bool Equals(object obj) |
|
bool op_Equality(Experience<T, TState, TAction> left, Experience<T, TState, TAction> right) |
|
bool op_Inequality(Experience<T, TState, TAction> left, Experience<T, TState, TAction> right) |
|
int GetHashCode() |
|
string ToString() |
|
void Deconstruct(TState State, TAction Action, T Reward, TState NextState, bool Done) |