LocalSamplingOptions
Options & Configuration · AiDotNet.Agentic.Models.Local
Controls how the next token is chosen from the model’s logits: temperature, top-k, top-p (nucleus), and an optional seed for reproducibility.
For Beginners
After the model says how likely each next word-piece is, these settings decide how to pick one. Low temperature = safe and repetitive; higher = more creative. Top-k (“only consider the best k options”) and top-p (“only consider the most likely options that together cover p% of the probability”) keep the choice from wandering into unlikely tokens.
How It Works
All values are nullable with sensible behavior when unset. A temperature of 0 (or less) selects
greedy decoding (always the highest-scoring token); higher temperatures increase randomness. Top-k and
top-p restrict sampling to the most likely tokens. These mirror the knobs exposed on
ChatOptions, which override these per request.
Constructors
| Constructor | Summary |
|---|---|
LocalSamplingOptions() |
Properties
| Property | Summary |
|---|---|
double? Temperature { get; set; } |
Gets or sets the sampling temperature. |
double? TopP { get; set; } |
Gets or sets the nucleus-sampling probability mass (0–1): only the most likely tokens whose cumulative probability reaches this value are considered. |
int? Seed { get; set; } |
Gets or sets the random seed for reproducible sampling. |
int? TopK { get; set; } |
Gets or sets the number of highest-probability tokens to consider. |