RecurrentPolicyAgent<T>

Models & Types · AiDotNet.Finance.Trading.Agents

A RECURRENT continuous policy for the portfolio-manager harness: an LSTM carries hidden state across the episode, so the policy conditions each action on the whole history it has seen — not just the fixed observation window an MLP policy is limited to. This is the recurrent-policy path for the greenfield trading-agent research: partial observability (regime, position history) is exactly where a recurrent policy should beat a memoryless one, and the experiment runner ranks that on the untouched holdout.

Trained on-policy with REINFORCE + a return baseline, back-propagating through time over each episode’s rollout (the natural fit for recurrence — on-policy methods learn from full sequences, no sequence-replay surgery). Exploration uses a fixed Gaussian σ, so the policy-gradient objective reduces to Σ advantageₜ · ½‖actionₜ − μₜ‖² / σ² — a numerically robust loss (no exp/log of parameters). Reuses the tape-trainable LSTM cell + Adam path proven by the DeepAR model. Implements the narrow IPortfolioAgent so it drops straight into PortfolioExperimentRunner.

Implements: IDisposable, IPortfolioAgent<T>

Constructors

Constructor Summary
RecurrentPolicyAgent(int stateDim, int actionDim, int hidden, double explorationSigma, double learningRate, double gamma, int seed)

Methods

Method Summary
T Train()
Vector<T> SelectAction(Vector<T> state, bool explore)
void Dispose() Releases the LSTM cell this agent owns (its pool-rented weights and registered engine state).
void ResetEpisode()
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)