LSPIOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for LSPI (Least-Squares Policy Iteration) agents.

Inheritance: ModelOptions → ReinforcementLearningOptions<T> → LSPIOptions<T>

For Beginners

LSPI is like repeatedly asking “what’s the best policy?” and “how good is it?” until the answers stop changing. Each iteration uses LSTD to evaluate the current policy, then improves it based on those evaluations.

Best for:

  • Batch reinforcement learning
  • Offline learning from fixed datasets
  • Sample-efficient policy learning
  • When you need guaranteed convergence

Not suitable for:

  • Online/streaming scenarios
  • Very large feature spaces
  • Continuous action spaces
  • Real-time learning requirements

How It Works

LSPI combines least-squares methods with policy iteration. It alternates between policy evaluation (using LSTDQ) and policy improvement, iteratively refining the policy until convergence.

Constructors

Constructor Summary
LSPIOptions()

Properties

Property Summary
double ConvergenceThreshold { get; set; } Weight change threshold for determining convergence.
double RegularizationParam { get; set; } Regularization parameter to prevent overfitting and ensure numerical stability.
int ActionSize { get; set; } Size of the action space (number of possible actions).
int FeatureSize { get; set; } Number of features in the state representation.
int MaxIterations { get; set; } Maximum number of policy iteration steps before stopping.