IMaskableAgent<T>

Interfaces · AiDotNet.Finance.Interfaces

An agent whose discrete action selection can be restricted to a legal subset.

Known implementations: FinRLAgent<T>, FinancialA2CAgent<T>, FinancialDQNAgent<T>, FinancialPPOAgent<T>

How It Works

Pre-shielding, not post-filtering. The safety-RL literature distinguishes two shapes: a post-shield replaces an unsafe action after the policy chose it, while a pre-shield supplies the legal set so the policy chooses within it. Only the second is sound for learning — a policy whose choice is overridden afterwards is trained toward actions it is then denied, so it learns from a payoff it can never realise. This interface is the pre-shield.

Opt-in, because most agents cannot use it. Masking is meaningful only over a discrete index set. Continuous policies (SAC, CQL, IQL, DDPG) emit a real-valued vector — often a Gaussian mean and log-sigma — where “legal action” is a feasible region, not a set of indices, and constraining that is projection rather than masking. Those agents deliberately do not implement this interface.

An unsupported mask is REFUSED, not dropped. A caller holding a non-null mask and an agent that does not implement this interface must throw, not fall back to unmasked selection. That fallback looks like graceful degradation and is not: the mask exists because some actions are unsafe or unrepresentable, so selecting without it returns an ILLEGAL action to a caller who asked for a legal one, with nothing in the result distinguishing the two. Only a null mask — meaning no restriction was asked for — permits the unmasked path. FinRLAgent<T> is the worked example: it forwards to a maskable inner agent and throws when its inner agent is continuous.

Why a separate interface rather than widening SelectAction. That method is overridden 61 times across this library — MuZero, Dreamer, Decision Transformer, the tabular agents, the bandits, the policy classes — and adding a parameter to the abstract declaration forces every one of them to change for a capability almost none can honour. The surveyed ecosystem reaches the same conclusion by a different route: sb3-contrib ships a separate MaskablePPO rather than altering PPO.predict.

Methods

Method Summary
Vector<T> SelectAction(Vector<T> state, bool training, bool[] legalActions) Selects an action restricted to the legal set.