SamplingOptions

Options & Configuration · AiDotNet.NeuralNetworks.Generation

Decoding/sampling configuration for autoregressive text generation (#1632 / #95). Centralises the knobs that were previously reimplemented per multimodal model (GPT4Vision/Blip/Flamingo each rolled their own temperature + softmax + sample loop).

How It Works

For Beginners: these control how the model picks the next token.

  • Temperature — randomness. 0 (or very small) = always pick the single most likely

token (greedy, deterministic). 1.0 = sample from the model’s raw distribution. >1 = flatter / more random; <1 = sharper / more focused.

  • TopK — only consider the K most-likely tokens (0 = no limit).
  • TopP (nucleus) — only consider the smallest set of tokens whose probabilities sum to

at least P (0 or ≥1 = no limit).

  • Seed — set for reproducible sampling; null uses the shared thread-safe RNG.

Constructors

Constructor Summary
SamplingOptions()

Properties

Property Summary
SamplingOptions Default { get; } Default sampling (temperature 1.0, no top-k/top-p).
SamplingOptions Greedy { get; } Greedy (argmax) decoding — deterministic.
bool IsGreedy { get; } True when these options request deterministic greedy decoding.
double Temperature { get; set; } Softmax temperature.
double TopP { get; set; } Nucleus threshold: keep the smallest token set whose cumulative probability ≥ TopP.
int TopK { get; set; } Keep only the K highest-logit tokens before sampling.
int? Seed { get; set; } Seed for reproducible sampling; null ⇒ shared thread-safe RNG.

Fields

Field Summary
double GreedyTemperatureEpsilon At/below this temperature, sampling collapses to greedy argmax (avoids divide-by-~0).