IMaskedActionEnvironment<T>
Interfaces · AiDotNet.Interfaces
An environment that can say which discrete actions are legal in the current state.
Implements: IEnvironment<T>
Known implementations: TradingEnvironment<T>
How It Works
Why the ENVIRONMENT owns legality. Whether an action can be taken is a property of the world, not of the policy: a chess position has legal moves, an account has structures it is cleared to trade. The agent is the wrong place to encode that, and a post-hoc filter applied after selection is worse still — the policy is then trained toward actions it is subsequently denied, so it learns from a payoff it can never realise.
This follows the established convention rather than inventing one. PettingZoo carries the
mask in the observation (observation["action_mask"]); Shimmy’s OpenSpiel environments and RLlib
carry it alongside the observation in the info/dict channel; OpenSpiel exposes legal_actions() on
the state. All of them put legality with the world and let the policy consult it. The one shape deliberately
NOT copied is folding the mask into the observation vector itself — that would change
ObservationSpaceDimension and therefore every agent’s input width, which is
exactly why RLlib uses a Dict space instead.
Opt-in, so nothing existing changes. This is a separate interface rather than a new member on
IEnvironment: an environment with no notion of illegal actions implements nothing, and the
60-odd agents that cannot use a mask are untouched.
What a consumer may do with the result. The unmasked path is permitted when there is no
restriction to honour — the environment does not implement this interface, or it does and
LegalActionMask is null. It is NOT permitted once a non-null mask is in
hand: an agent that cannot apply one must refuse, because selecting unmasked returns an illegal action
while the caller believes it asked for a legal one. See
IMaskableAgent.
Convention for the info dictionary. Implementations SHOULD also publish the same array under
action_mask in the
dictionary returned by Step, so a consumer
that only has the step result can read it without holding the environment reference. The property is the
authority; the info entry is a mirror. The property also covers the case the info channel cannot:
Reset returns only an observation, so an agent choosing its FIRST action has
no step result to read.
Properties
| Property | Summary |
|---|---|
bool[] LegalActionMask { get; } |
One flag per discrete action: true where the action may be selected in the current state, false where it may not. |