GenerationRequest<T>
Models & Types · AiDotNet.Serving.ContinuousBatching
Represents a request for text generation.
Constructors
| Constructor | Summary |
|---|---|
GenerationRequest() |
Properties
| Property | Summary |
|---|---|
Action<int> OnTokenGenerated { get; set; } |
Callback for streaming tokens. |
IReadOnlyDictionary<int, float> LogitBias { get; set; } |
Optional per-token additive logit bias (OpenAI logit_bias): token id -> bias added to the token’s logit before sampling. |
ITokenConstraint Constraint { get; set; } |
Optional structured-output constraint (JSON, regex, grammar, or choice) that restricts which tokens may be emitted so the output conforms to a required format. |
List<int> PromptTokenIds { get; set; } |
Token IDs of the prompt. |
List<int> StopTokenIds { get; set; } |
Additional stop token IDs. |
bool IncludeLogProbs { get; set; } |
Whether to record per-token log-probabilities during generation (OpenAI logprobs). |
bool UseBeamSearch { get; set; } |
Whether to use beam search. |
float FrequencyPenalty { get; set; } |
OpenAI frequency_penalty (default 0): each token’s logit is reduced by this value times the number of times it has already appeared in the text so far, discouraging verbatim repetition. |
float MinP { get; set; } |
Min-p sampling threshold: drop tokens below this fraction of the top token’s probability (0 = disabled). |
float PresencePenalty { get; set; } |
OpenAI presence_penalty (default 0): a token’s logit is reduced by this value once if it has appeared at all in the text so far, encouraging the model to introduce new tokens/topics. |
float RepetitionPenalty { get; set; } |
Repetition penalty (1.0 = no penalty). |
float Temperature { get; set; } |
Temperature for sampling (higher = more random). |
float TopP { get; set; } |
Top-p (nucleus) sampling threshold. |
int MaxNewTokens { get; set; } |
Maximum number of new tokens to generate. |
int NumBeams { get; set; } |
Number of beams for beam search. |
int Priority { get; set; } |
Priority for scheduling (higher = more important). |
int TopK { get; set; } |
Top-k sampling (0 = disabled). |
int TopLogProbs { get; set; } |
How many of the most likely alternative tokens to record per position (OpenAI top_logprobs, 0-20). |
int? EosTokenId { get; set; } |
Optional end-of-sequence token id for this request. |
int? Seed { get; set; } |
Optional RNG seed for reproducible sampling. |
int? SpeculationDepth { get; set; } |
Optional per-request speculative-decoding draft depth. |
object UserContext { get; set; } |
Optional user context. |
string AdapterId { get; set; } |
Optional multi-LoRA adapter name to activate for this request (S-LoRA-style serving). |