FinancialA2CAgent<T>

Models & Types · AiDotNet.Finance.Trading.Agents

Financial Advantage Actor-Critic (A2C) agent for fast trading policy learning.

Inheritance: ReinforcementLearningAgentBase<T> → TradingAgentBase<T> → FinancialA2CAgent<T>

Implements: IGeneratedParameterRegistrar<T>, IGradientComputable<T, Vector<T>, Vector<T>>, IMaskableAgent<T>

For Beginners

The A2C (Advantage Actor-Critic) trading agent uses two neural networks working together: an “actor” that decides what trades to make, and a “critic” that evaluates how good those decisions are. The advantage of A2C is that it learns quickly because the critic provides immediate feedback to the actor after each trade, rather than waiting for the end result. It is well-suited for fast-paced trading environments where quick adaptation is important.

How It Works

This is an n-step, on-policy categorical actor-critic: the critic is fit to the n-step return over the rollout (NSteps steps, or fewer when an episode ends early), not to a one-step TD target, so raising NSteps trades variance for bias in the usual direction. Collect actions with SelectAction(state, training: true) and store them before changing the actor. Each update consumes the current rollout once; it never replays transitions from an older actor. Agent parameter/gradient/checkpoint updates discard pending behavior. Ordinary writes to stable live actor tensors are also detected by storage identity and mutation version.

Custom layers with opaque/detached parameter storage, retained raw writable spans and mutations under tensor inference mode must use the agent’s explicit update boundary. Do not mutate a policy concurrently with collection. Public collection accepts each sampled action object exactly once for the state values used to select it. Caller-created, copied, foreign, greedy, changed and stale actions are rejected. The explicit supervised Train(state, target) API instead isolates its labelled transition from pending on-policy data. Pending behavior and selection stamps are runtime-only and are not restored from checkpoints.

Example

// Define actor and critic architectures for A2C trading (30 state features, 3 actions)
var actorArch = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 30, outputSize: 3);
var criticArch = new NeuralNetworkArchitecture<double>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 30, outputSize: 1);
            
// Create A2C agent for fast-adapting trading policy
var options = new TradingAgentOptions<double>();
var model = new FinancialA2CAgent<double>(actorArch, criticArch, options);

Constructors

Constructor Summary
FinancialA2CAgent(NeuralNetworkArchitecture<T> actorArchitecture, NeuralNetworkArchitecture<T> criticArchitecture, TradingAgentOptions<T> options) Initializes a new instance of the FinancialA2CAgent class.

Properties

Property Summary
int FeatureCount { get; }
int PendingRolloutLength { get; } Transitions collected since the last policy update.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata() Executes GetModelMetadata for the FinancialA2CAgent.
ModelOptions GetOptions()
T Train()
Vector<T> ComputeGradients(Vector<T> input, Vector<T> target, ILossFunction<T> lossFunction) Executes ComputeGradients for the FinancialA2CAgent.
Vector<T> SelectAction(Vector<T> state, bool training)
Vector<T> SelectAction(Vector<T> state, bool training, bool[] legalActions)
byte[] Serialize() Auto-generated common model serialization surface.
void ApplyGradients(Vector<T> gradients, T learningRate) Executes ApplyGradients for the FinancialA2CAgent.
void Deserialize(byte[] data) Auto-generated common model deserialization surface.
void LoadModel(string filepath) Executes LoadModel for the FinancialA2CAgent.
void SaveModel(string filepath) Executes SaveModel for the FinancialA2CAgent.
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done)
void StoreExperience(Vector<T> state, Vector<T> action, T reward, Vector<T> nextState, bool done, bool[] nextLegalActions) Copies one current-policy transition into the bounded pending rollout.