ITokenConstraint

Interfaces · AiDotNet.Agentic.Models.Local

Restricts which tokens the local engine may generate next, enabling constrained decoding: the model can only emit tokens the constraint permits, so the output is guaranteed to satisfy a structure (a fixed vocabulary, a grammar, a JSON shape) rather than merely being asked to in the prompt.

Known implementations: AllowedTokenSetConstraint, FiniteStateTokenConstraint

For Beginners

Normally a model can pick any next word-piece. A constraint is a gate that only lets through the choices that keep the answer valid — for example, only digits when you want a number, or only tokens that continue well-formed JSON. Because the gate is applied while generating, the result is always valid by construction, not just “usually” valid.

How It Works

At each step the engine calls AllowedNextTokens with the tokens generated so far. Returning null means “no restriction this step”; returning a set restricts sampling to exactly those token ids; returning an empty set tells the engine to stop (nothing valid can follow). This is the foundation for local structured output and tool-calling — capabilities cloud models approximate with prompting but cannot guarantee, and which a local engine can enforce at the logits because it controls decoding.

Methods

Method Summary
IReadOnlyCollection<int> AllowedNextTokens(IReadOnlyList<int> generatedTokenIds) Returns the token ids permitted as the next token given what has been generated so far.