Reinforcement Learning

99 public types in this namespace, organized by kind.

Models & Types (81)

Type Summary
A2CAgent<T> Advantage Actor-Critic (A2C) agent for reinforcement learning.
A3CAgent<T> Asynchronous Advantage Actor-Critic (A3C) agent for reinforcement learning.
BetaPolicy<T> Policy using Beta distribution for bounded continuous action spaces.
BoltzmannExploration<T> Boltzmann (softmax) exploration with temperature-based action selection.
CQLAgent<T> Conservative Q-Learning (CQL) agent for offline reinforcement learning.
CartPoleEnvironment<T> Classic CartPole-v1 environment for reinforcement learning.
ContinuousPolicy<T> Policy for continuous action spaces using a neural network to output Gaussian parameters.
DDPGAgent<T> Deep Deterministic Policy Gradient (DDPG) agent for continuous control.
DQNAgent<T> Deep Q-Network (DQN) agent for reinforcement learning.
DecisionTransformerAgent<T> Decision Transformer agent for offline reinforcement learning.
DeterministicBanditEnvironment<T> A deterministic multi-armed bandit environment for testing purposes.
DeterministicPolicy<T> Deterministic policy for continuous action spaces.
DiscretePolicy<T> Policy for discrete action spaces using a neural network to output action logits.
DoubleDQNAgent<T> Double Deep Q-Network (Double DQN) agent for reinforcement learning.
DoubleQLearningAgent<T> Double Q-Learning agent using two Q-tables to reduce overestimation bias.
DreamerAgent<T> Dreamer agent (Hafner et al.
DuelingDQNAgent<T> Dueling Deep Q-Network agent for reinforcement learning.
DynaQAgent<T> Dyna-Q agent combining learning and planning using a learned model.
DynaQPlusAgent<T> Dyna-Q+ agent with exploration bonus for handling changing environments.
EpsilonGreedyBanditAgent<T> Epsilon-Greedy Multi-Armed Bandit agent.
EpsilonGreedyExploration<T> Epsilon-greedy exploration: with probability epsilon, select random action.
EveryVisitMonteCarloAgent<T> Every-Visit Monte Carlo agent that updates all visits to states in an episode.
ExpectedSARSAAgent<T> Expected SARSA agent using tabular methods.
Experience<T> Simplified Experience record for Vector-based states and actions.
Experience<T, TState, TAction> Represents a single experience tuple (s, a, r, s’, done) for reinforcement learning.
FirstVisitMonteCarloAgent<T> First-Visit Monte Carlo agent for episodic tasks.
GaussianNoiseExploration<T> Gaussian noise exploration for continuous action spaces.
GradientBanditAgent<T> Gradient Bandit agent using softmax action preferences.
IQLAgent<T> Implicit Q-Learning (IQL) agent for offline reinforcement learning.
InterleavedCountParameterSource<T> A pair of per-arm integer counters exposed as one INTERLEAVED parameter vector: [first[0], second[0], first[1], second[1], ...].
LSPIAgent<T> LSPI (Least-Squares Policy Iteration) agent using iterative policy improvement with LSTDQ.
LSTDAgent<T> LSTD (Least-Squares Temporal Difference) agent using direct solution for value function weights.
LinearQLearningAgent<T> Linear Q-Learning agent using linear function approximation.
LinearSARSAAgent<T> Linear SARSA agent using linear function approximation with on-policy learning.
MADDPGAgent<T> Multi-Agent Deep Deterministic Policy Gradient (MADDPG) agent for cooperative and competitive multi-agent reinforcement learning with continuous action spaces.
MatrixParameterSource<T> Exposes a linear agent’s weight matrix as a parameter source, row-major.
MixedPolicy<T> Policy for environments with both discrete and continuous action spaces.
ModifiedPolicyIterationAgent<T> Modified Policy Iteration agent - hybrid of Policy Iteration and Value Iteration.
MonteCarloExploringStartsAgent<T> Monte Carlo Exploring Starts agent for reinforcement learning.
MuZeroAgent<T> MuZero agent combining tree search with learned models.
MultiModalPolicy<T> Multi-modal policy using mixture of Gaussians for complex action distributions.
NStepQLearningAgent<T> N-step Q-Learning agent using multi-step off-policy returns.
NStepSARSAAgent<T> N-step SARSA agent using multi-step bootstrapping.
NoExploration<T> No exploration - always use the policy’s action directly (greedy).
OffPolicyMonteCarloAgent<T> Off-Policy Monte Carlo Control agent with weighted importance sampling.
OnPolicyMonteCarloAgent<T> On-Policy Monte Carlo Control agent with epsilon-greedy exploration.
OrnsteinUhlenbeckNoise<T> Ornstein-Uhlenbeck process noise for temporally correlated exploration.
PPOAgent<T> Proximal Policy Optimization (PPO) agent for reinforcement learning.
PolicyIterationAgent<T> Policy Iteration agent for reinforcement learning using dynamic programming.
PrioritizedReplayBuffer<T> Prioritized experience replay buffer for reinforcement learning.
PrioritizedSweepingAgent<T> Prioritized Sweeping agent that focuses planning on high-priority state-actions.
QLambdaAgent<T> Q(lambda) agent that combines Q-learning with eligibility traces for faster credit assignment in tabular reinforcement learning environments.
QMIXAgent<T> QMIX (Rashid et al.
QTableEntriesParameterSource<T> Exposes a Q-table as the entries it ACTUALLY holds, rather than padding each state out to the full action count.
QTableParameterSource<T> Exposes a tabular agent’s Q-table as a parameter source, so it can be registered like a network instead of every agent hand-writing the same flatten-and-refill loops.
REINFORCEAgent<T> REINFORCE (Monte Carlo Policy Gradient) agent for reinforcement learning.
RainbowDQNAgent<T> Rainbow DQN agent combining six extensions to DQN.
RandomNetworkDistillation<T> Random Network Distillation (Burda et al.
ReplayBuffer<T> A buffer for storing and replaying experiences in reinforcement learning.
SACAgent<T> Soft Actor-Critic (SAC) agent for continuous control reinforcement learning.
SARSAAgent<T> SARSA (State-Action-Reward-State-Action) agent using tabular methods.
SARSALambdaAgent<T> SARSA(lambda) agent that combines on-policy SARSA control with eligibility traces for faster credit assignment while respecting the current exploration policy.
SequenceContext<T> Context window for sequence modeling in Decision Transformer.
TD3Agent<T> Twin Delayed Deep Deterministic Policy Gradient (TD3) agent for continuous control.
TRPOAgent<T> Trust Region Policy Optimization (TRPO) agent for reinforcement learning (Schulman et al.
TabularActorCriticAgent<T> Tabular Actor-Critic agent combining policy and value learning.
TabularQLearningAgent<T> Tabular Q-Learning agent using lookup table for Q-values.
ThompsonSamplingAgent<T> Thompson Sampling (Bayesian) Multi-Armed Bandit agent.
ThompsonSamplingExploration<T> Thompson Sampling (Bayesian) exploration for discrete action spaces.
Trajectory<T> Represents a trajectory of experience for on-policy RL algorithms (PPO, A2C, etc.).
TransitionData<T> Helper class for serializing model transition data.
UCBBanditAgent<T> Upper Confidence Bound (UCB) Multi-Armed Bandit agent.
UniformReplayBuffer<T, TState, TAction> A replay buffer that samples experiences uniformly at random.
UpperConfidenceBoundExploration<T> Upper Confidence Bound (UCB) exploration for discrete action spaces.
ValueIterationAgent<T> Value Iteration agent for reinforcement learning using dynamic programming.
ValueTableParameterSource<T> Exposes a state-value table (Dictionary<string, T>) as a parameter source, for the agents that learn V(s) rather than Q(s, a).
VectorParameterSource<T> Exposes a plain Vector of learned values – a bandit’s action preferences, for instance – as a parameter source.
WatkinsQLambdaAgent<T> Watkins’s Q(lambda) agent that combines Q-learning with eligibility traces but cuts traces when an exploratory (non-greedy) action is taken, ensuring convergence to the optimal policy.
WorkerNetworks<T> Worker-local networks for A3C agent.
WorldModelsAgent<T> World Models agent (Ha & Schmidhuber 2018): a VAE (V), an MDN-RNN (M) and a CMA-ES-evolved controller (C) trained inside M’s dream.
WorldModelsTrainingCycle Where a WorldModelsAgent stands in its V -> M -> C training cycle.

Base Classes (4)

Type Summary
DeepReinforcementLearningAgentBase<T> Base class for deep reinforcement learning agents that use neural networks as function approximators.
ExplorationStrategyBase<T> Abstract base class for exploration strategy implementations.
PolicyBase<T> Abstract base class for policy implementations.
ReinforcementLearningAgentBase<T> Base class for all reinforcement learning agents, providing common functionality and structure.

Interfaces (4)

Type Summary
IExplorationStrategy<T> Interface for exploration strategies used by policies.
IIntrinsicRewardModule<T> Produces an intrinsic (curiosity) reward for a state, driving a reinforcement-learning agent toward novel states even when the environment’s extrinsic reward is sparse.
IPolicy<T> Core interface for RL policies - defines how to select actions.
IReplayBuffer<T, TState, TAction> Interface for experience replay buffers used in reinforcement learning.

Enums (1)

Type Summary
WorldModelsTrainingStage The stage of the World Models training cycle that the next Train call runs.

Options & Configuration (7)

Type Summary
BetaPolicyOptions<T> Configuration options for Beta distribution policies: BetaPolicy’s #ctor constructor builds its network from them.
ContinuousPolicyOptions<T> Configuration options for continuous action space policies in reinforcement learning.
DeterministicPolicyOptions<T> Configuration options for deterministic policies: DeterministicPolicy’s #ctor constructor builds its network from them.
DiscretePolicyOptions<T> Configuration options for discrete action space policies in reinforcement learning.
MixedPolicyOptions<T> Configuration options for mixed discrete and continuous policies: MixedPolicy’s #ctor constructor builds its networks from them.
MultiModalPolicyOptions<T> Configuration options for multi-modal mixture-of-Gaussians policies: MultiModalPolicy’s #ctor constructor builds its network from them.
ReinforcementLearningOptions<T> Configuration options for reinforcement learning agents.

Helpers & Utilities (2)

Type Summary
EnvironmentInfoKeys Keys of the info dictionary returned by Step.
MCTSNode<T> Monte Carlo Tree Search (MCTS) node for MuZero agent.