SACOptions<T>
Options & Configuration · AiDotNet.Models.Options
Configuration options for Soft Actor-Critic (SAC) agents.
Inheritance: ModelOptions → SACOptions<T>
For Beginners
SAC is one of the best algorithms for continuous control (like robot movement).
Key innovations:
- Maximum Entropy: Encourages exploration by being “random on purpose”
- Off-Policy: Learns from old experiences (sample efficient)
- Twin Q-Networks: Uses two Q-functions to prevent overestimation
- Automatic Tuning: Adjusts exploration automatically
Think of it like learning to drive while staying diverse in your driving style - you don’t just learn one way to drive, you stay flexible and adaptable.
Used by: Robotic manipulation, dexterous control, autonomous systems
How It Works
SAC is a state-of-the-art off-policy actor-critic algorithm that combines maximum entropy RL with stable off-policy learning. It’s particularly effective for continuous control tasks and is known for excellent sample efficiency and robustness.
Constructors
| Constructor | Summary |
|---|---|
SACOptions() |
Creates options with the default values. |
Properties
| Property | Summary |
|---|---|
ILossFunction<T> QLossFunction { get; set; } |
Loss function for Q-networks (typically MSE). |
List<int> PolicyHiddenLayers { get; set; } |
Hidden layer sizes for policy network. |
List<int> QHiddenLayers { get; set; } |
Hidden layer sizes for Q-networks. |
T AlphaLearningRate { get; set; } |
Learning rate for temperature parameter (alpha). |
T DiscountFactor { get; set; } |
Discount factor (gamma) for future rewards. |
T InitialTemperature { get; set; } |
Initial temperature (alpha) for entropy regularization. |
T PolicyLearningRate { get; set; } |
Learning rate for policy network. |
T QLearningRate { get; set; } |
Learning rate for Q-networks. |
T TargetUpdateTau { get; set; } |
Soft target update coefficient (tau). |
bool AutoTuneTemperature { get; set; } |
Whether to automatically tune the temperature parameter. |
double? TargetEntropy { get; set; } |
Target entropy for automatic temperature tuning. |
int ActionSize { get; set; } |
Size of the continuous action space. |
int BatchSize { get; set; } |
Mini-batch size for training. |
int GradientSteps { get; set; } |
Number of gradient steps per environment step. |
int ReplayBufferSize { get; set; } |
Capacity of the experience replay buffer. |
int StateSize { get; set; } |
Size of the state observation space. |
int WarmupSteps { get; set; } |
Number of warmup steps before starting training. |