SACOptions<T>

Options & Configuration · AiDotNet.Models.Options

Configuration options for Soft Actor-Critic (SAC) agents.

Inheritance: ModelOptions → SACOptions<T>

For Beginners

SAC is one of the best algorithms for continuous control (like robot movement).

Key innovations:

  • Maximum Entropy: Encourages exploration by being “random on purpose”
  • Off-Policy: Learns from old experiences (sample efficient)
  • Twin Q-Networks: Uses two Q-functions to prevent overestimation
  • Automatic Tuning: Adjusts exploration automatically

Think of it like learning to drive while staying diverse in your driving style - you don’t just learn one way to drive, you stay flexible and adaptable.

Used by: Robotic manipulation, dexterous control, autonomous systems

How It Works

SAC is a state-of-the-art off-policy actor-critic algorithm that combines maximum entropy RL with stable off-policy learning. It’s particularly effective for continuous control tasks and is known for excellent sample efficiency and robustness.

Constructors

Constructor Summary
SACOptions() Creates options with the default values.

Properties

Property Summary
ILossFunction<T> QLossFunction { get; set; } Loss function for Q-networks (typically MSE).
List<int> PolicyHiddenLayers { get; set; } Hidden layer sizes for policy network.
List<int> QHiddenLayers { get; set; } Hidden layer sizes for Q-networks.
T AlphaLearningRate { get; set; } Learning rate for temperature parameter (alpha).
T DiscountFactor { get; set; } Discount factor (gamma) for future rewards.
T InitialTemperature { get; set; } Initial temperature (alpha) for entropy regularization.
T PolicyLearningRate { get; set; } Learning rate for policy network.
T QLearningRate { get; set; } Learning rate for Q-networks.
T TargetUpdateTau { get; set; } Soft target update coefficient (tau).
bool AutoTuneTemperature { get; set; } Whether to automatically tune the temperature parameter.
double? TargetEntropy { get; set; } Target entropy for automatic temperature tuning.
int ActionSize { get; set; } Size of the continuous action space.
int BatchSize { get; set; } Mini-batch size for training.
int GradientSteps { get; set; } Number of gradient steps per environment step.
int ReplayBufferSize { get; set; } Capacity of the experience replay buffer.
int StateSize { get; set; } Size of the state observation space.
int WarmupSteps { get; set; } Number of warmup steps before starting training.