DraftResult<T>
Models & Types · AiDotNet.Inference.SpeculativeDecoding
Result of draft token generation returned by an IDraftModel: the guessed tokens and the
probability distributions the draft assigned to them (used by the target model to accept/reject drafts).
For Beginners
When a draft model guesses the next few tokens, it returns them here along with how confident it was about each. The main model uses that information to decide which guesses to keep.
Constructors
| Constructor | Summary |
|---|---|
DraftResult() |
Properties
| Property | Summary |
|---|---|
Matrix<T> Probabilities { get; set; } |
Gets the probability distributions for each draft position. |
Vector<T> TokenProbabilities { get; set; } |
Gets the sampled token probabilities (p(token) for each drafted token). |
Vector<int> Tokens { get; set; } |
Gets the generated draft tokens. |
int NumTokens { get; } |
Gets the number of draft tokens generated. |