IMaskedActionEnvironment<T>

Interfaces · AiDotNet.Interfaces

An environment that can say which discrete actions are legal in the current state.

Implements: IEnvironment<T>

Known implementations: TradingEnvironment<T>

How It Works

Why the ENVIRONMENT owns legality. Whether an action can be taken is a property of the world, not of the policy: a chess position has legal moves, an account has structures it is cleared to trade. The agent is the wrong place to encode that, and a post-hoc filter applied after selection is worse still — the policy is then trained toward actions it is subsequently denied, so it learns from a payoff it can never realise.

This follows the established convention rather than inventing one. PettingZoo carries the mask in the observation (observation["action_mask"]); Shimmy’s OpenSpiel environments and RLlib carry it alongside the observation in the info/dict channel; OpenSpiel exposes legal_actions() on the state. All of them put legality with the world and let the policy consult it. The one shape deliberately NOT copied is folding the mask into the observation vector itself — that would change ObservationSpaceDimension and therefore every agent’s input width, which is exactly why RLlib uses a Dict space instead.

Opt-in, so nothing existing changes. This is a separate interface rather than a new member on IEnvironment: an environment with no notion of illegal actions implements nothing, and the 60-odd agents that cannot use a mask are untouched.

What a consumer may do with the result. The unmasked path is permitted when there is no restriction to honour — the environment does not implement this interface, or it does and LegalActionMask is null. It is NOT permitted once a non-null mask is in hand: an agent that cannot apply one must refuse, because selecting unmasked returns an illegal action while the caller believes it asked for a legal one. See IMaskableAgent.

Convention for the info dictionary. Implementations SHOULD also publish the same array under action_mask in the dictionary returned by Step, so a consumer that only has the step result can read it without holding the environment reference. The property is the authority; the info entry is a mirror. The property also covers the case the info channel cannot: Reset returns only an observation, so an agent choosing its FIRST action has no step result to read.

Properties

Property Summary
bool[] LegalActionMask { get; } One flag per discrete action: true where the action may be selected in the current state, false where it may not.