TokenSampler<T>

Models & Types · AiDotNet.Agentic.Models.Local

Selects the next token id from a logits vector according to LocalSamplingOptions: greedy (argmax) when temperature is zero, otherwise temperature-scaled softmax sampling restricted by optional top-k and top-p filters.

For Beginners

This is the “dice roll” step. With temperature 0 it isn’t a roll at all — it just takes the single most likely token. Otherwise it turns the scores into probabilities, optionally throws away the unlikely options (top-k / top-p), and then picks one at random in proportion to how likely each is.

How It Works

The sampler owns a single Random (seeded from Seed when provided), so a fixed seed yields a reproducible token stream. Logits are read through ToDouble, so the same code path serves Single and Double models.

Constructors

Constructor Summary
TokenSampler(int? seed) Initializes a new sampler.

Methods

Method Summary
int Sample(Vector<T> logits, LocalSamplingOptions options, IReadOnlyCollection<int> allowedTokenIds) Chooses the next token id from the supplied logits, optionally restricted to an allowed set (constrained decoding).