OrnsteinUhlenbeckNoise<T>
Models & Types · AiDotNet.ReinforcementLearning.Policies.Exploration
Ornstein-Uhlenbeck process noise for temporally correlated exploration. Commonly used in DDPG and other continuous control algorithms. Process equation: dx = θ(μ - x)dt + σdW
Inheritance: ExplorationStrategyBase<T> → OrnsteinUhlenbeckNoise<T>
Example
var random = new Random(42);
var state = new Vector<double>(new double[] { 0.1, -0.4, 0.7, 0.2 });
// Create Ornstein-Uhlenbeck noise for temporally correlated exploration
var ouNoise = new OrnsteinUhlenbeckNoise<double>(actionSize: 2, theta: 0.15, sigma: 0.2);
// Add temporally correlated noise to continuous actions
var policyAction = new Vector<double>(new double[] { 0.5, -0.3 });
var noisyAction = ouNoise.GetExplorationAction(state, policyAction, actionSpaceSize: 2, random);
Constructors
| Constructor | Summary |
|---|---|
OrnsteinUhlenbeckNoise() |
Initializes a new instance of the Ornstein-Uhlenbeck noise exploration. |
OrnsteinUhlenbeckNoise(int actionSize, double theta, double sigma, double mu, double dt) |
Methods
| Method | Summary |
|---|---|
Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random) |
Applies Ornstein-Uhlenbeck noise to the policy action. |
void Reset() |
Resets the noise state to zero. |
void Update() |
Updates internal parameters (no-op for OU noise as it self-regulates). |