RecurrentPolicyAgent<T>
Models & Types · AiDotNet.Finance.Trading.Agents
A RECURRENT continuous policy for the portfolio-manager harness: an LSTM carries hidden state across the episode, so the policy conditions each action on the whole history it has seen — not just the fixed observation window an MLP policy is limited to. This is the recurrent-policy path for the greenfield trading-agent research: partial observability (regime, position history) is exactly where a recurrent policy should beat a memoryless one, and the experiment runner ranks that on the untouched holdout.
Trained on-policy with REINFORCE + a return baseline, back-propagating through time over each episode’s
rollout (the natural fit for recurrence — on-policy methods learn from full sequences, no sequence-replay
surgery). Exploration uses a fixed Gaussian σ, so the policy-gradient objective reduces to
Σ advantageₜ · ½‖actionₜ − μₜ‖² / σ² — a numerically robust loss (no exp/log of parameters). Reuses
the tape-trainable LSTM cell + Adam path proven by the DeepAR model. Implements the narrow
IPortfolioAgent so it drops straight into PortfolioExperimentRunner.
Implements: IDisposable, IPortfolioAgent<T>
Constructors
| Constructor | Summary |
|---|---|
RecurrentPolicyAgent(int stateDim, int actionDim, int hidden, double explorationSigma, double learningRate, double gamma, int seed) |
Methods
| Method | Summary |
|---|---|
T Train() |
|
Vector<T> SelectAction(Vector<T> state, bool explore) |
|
void Dispose() |
Releases the LSTM cell this agent owns (its pool-rented weights and registered engine state). |
void ResetEpisode() |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |