OrnsteinUhlenbeckNoise<T>

Models & Types · AiDotNet.ReinforcementLearning.Policies.Exploration

Ornstein-Uhlenbeck process noise for temporally correlated exploration. Commonly used in DDPG and other continuous control algorithms. Process equation: dx = θ(μ - x)dt + σdW

Inheritance: ExplorationStrategyBase<T> → OrnsteinUhlenbeckNoise<T>

Example

var random = new Random(42);
var state = new Vector<double>(new double[] { 0.1, -0.4, 0.7, 0.2 });
// Create Ornstein-Uhlenbeck noise for temporally correlated exploration
var ouNoise = new OrnsteinUhlenbeckNoise<double>(actionSize: 2, theta: 0.15, sigma: 0.2);
            
// Add temporally correlated noise to continuous actions
var policyAction = new Vector<double>(new double[] { 0.5, -0.3 });
var noisyAction = ouNoise.GetExplorationAction(state, policyAction, actionSpaceSize: 2, random);

Constructors

Constructor Summary
OrnsteinUhlenbeckNoise() Initializes a new instance of the Ornstein-Uhlenbeck noise exploration.
OrnsteinUhlenbeckNoise(int actionSize, double theta, double sigma, double mu, double dt)

Methods

Method Summary
Vector<T> GetExplorationAction(Vector<T> state, Vector<T> policyAction, int actionSpaceSize, Random random) Applies Ornstein-Uhlenbeck noise to the policy action.
void Reset() Resets the noise state to zero.
void Update() Updates internal parameters (no-op for OU noise as it self-regulates).