FinancialA2CAgent<T>
Models & Types · AiDotNet.Finance.Trading.Agents
Financial Advantage Actor-Critic (A2C) agent for fast trading policy learning.
Inheritance: ReinforcementLearningAgentBase<T> → TradingAgentBase<T> → FinancialA2CAgent<T>
Implements: IGeneratedParameterRegistrar<T>, IGradientComputable<T, Vector<T>, Vector<T>>, IMaskableAgent<T>
For Beginners
The A2C (Advantage Actor-Critic) trading agent uses two neural networks working together: an “actor” that decides what trades to make, and a “critic” that evaluates how good those decisions are. The advantage of A2C is that it learns quickly because the critic provides immediate feedback to the actor after each trade, rather than waiting for the end result. It is well-suited for fast-paced trading environments where quick adaptation is important.
How It Works
This is an n-step, on-policy categorical actor-critic: the critic is fit to the n-step return over the
rollout (NSteps steps, or fewer when an episode ends early),
not to a one-step TD target, so raising NSteps trades variance for bias in the usual direction. Collect
actions with
SelectAction(state, training: true) and store them before changing the actor. Each update
consumes the current rollout once; it never replays transitions from an older actor.
Agent parameter/gradient/checkpoint updates discard pending behavior. Ordinary writes to stable
live actor tensors are also detected by storage identity and mutation version.
Custom layers with opaque/detached parameter storage, retained raw writable spans and mutations
under tensor inference mode must use the agent’s explicit update boundary. Do not mutate a policy
concurrently with collection. Public collection accepts each sampled action object exactly once
for the state values used to select it. Caller-created, copied, foreign, greedy, changed and stale
actions are rejected. The explicit supervised Train(state, target) API instead isolates its
labelled transition from pending on-policy data. Pending behavior and selection stamps are
runtime-only and are not restored from checkpoints.
Example
// Define actor and critic architectures for A2C trading (30 state features, 3 actions)
var actorArch = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputSize: 30, outputSize: 3);
var criticArch = new NeuralNetworkArchitecture<double>(
inputType: InputType.OneDimensional,
taskType: NeuralNetworkTaskType.Regression,
inputSize: 30, outputSize: 1);
// Create A2C agent for fast-adapting trading policy
var options = new TradingAgentOptions<double>();
var model = new FinancialA2CAgent<double>(actorArch, criticArch, options);
Constructors
| Constructor | Summary |
|---|---|
FinancialA2CAgent(NeuralNetworkArchitecture<T> actorArchitecture, NeuralNetworkArchitecture<T> criticArchitecture, TradingAgentOptions<T> options) |
Initializes a new instance of the FinancialA2CAgent class. |
Properties
| Property | Summary |
|---|---|
int FeatureCount { get; } |
|
int PendingRolloutLength { get; } |
Transitions collected since the last policy update. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Executes GetModelMetadata for the FinancialA2CAgent. |
ModelOptions GetOptions() |
|
T Train() |
|
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction) |
Executes ComputeGradients for the FinancialA2CAgent. |
Vector<T> SelectAction(Vector<T> state, bool training) |
|
Vector<T> SelectAction(Vector<T> state, bool training, bool[] legalActions) |
|
byte[] Serialize() |
Auto-generated common model serialization surface. |
void ApplyGradients(Vector<T> gradients, T learningRate) |
Executes ApplyGradients for the FinancialA2CAgent. |
void Deserialize(byte[] data) |
Auto-generated common model deserialization surface. |
void LoadModel(string filepath) |
Executes LoadModel for the FinancialA2CAgent. |
void SaveModel(string filepath) |
Executes SaveModel for the FinancialA2CAgent. |
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done) |
|
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done, bool[] nextLegalActions) |
Copies one current-policy transition into the bounded pending rollout. |