ITokenConstraint
Interfaces · AiDotNet.Serving.StructuredOutput
A per-sequence decoding constraint that restricts which vocabulary tokens may be emitted next, enabling structured / guided output (JSON, regex, grammar-constrained generation).
Known implementations: JsonGrammarConstraint, RegexTokenConstraint, TokenFsmConstraint
For Beginners
This is how “give me valid JSON” or “answer only yes/no” is enforced. At every step the model still produces scores for all tokens; the constraint blanks out the ones that would break the required format so the model is forced to stay on the rails, while still choosing freely among the tokens that keep the output valid.
How It Works
A constraint is a small state machine over generated tokens. Before each sampling step the engine
calls ApplyMask to forbid tokens that would violate the target language; after a token
is chosen it calls Accept to advance the state. The constraint is stateful and belongs
to exactly one sequence — never share an instance across concurrent sequences.
Implementers: this interface is new in the serving layer and defines both
IsComplete (the output so far is a complete valid instance) and IsTerminal
(complete AND no valid non-EOS continuation remains — where the engine stops). A custom constraint must
implement both; a common pattern is IsTerminal => IsComplete && !HasNonEosContinuation.
The built-in constraints (JSON grammar/schema, regex, token FSM) are the reference implementations.
Properties
| Property | Summary |
|---|---|
bool IsComplete { get; } |
Gets a value indicating whether the generated text so far is accepting — a complete valid instance of the target language. |
bool IsTerminal { get; } |
Gets a value indicating whether the constraint is terminal: the generated text so far is a complete valid instance of the target language AND no valid non-EOS continuation remains, so the engine may stop the sequence. |
Methods
| Method | Summary |
|---|---|
void Accept(int tokenId) |
Advances the constraint state after tokenId has been committed to the sequence. |
void ApplyMask(Span<float> logits) |
Masks tokens that are not permitted as the next token in-place: sets logits[i] = for every token id i that would violate the constraint given the tokens accepted so far. |