DraftResult<T>

Models & Types · AiDotNet.Inference.SpeculativeDecoding

Result of draft token generation returned by an IDraftModel: the guessed tokens and the probability distributions the draft assigned to them (used by the target model to accept/reject drafts).

For Beginners

When a draft model guesses the next few tokens, it returns them here along with how confident it was about each. The main model uses that information to decide which guesses to keep.

Constructors

Constructor Summary
DraftResult()

Properties

Property Summary
Matrix<T> Probabilities { get; set; } Gets the probability distributions for each draft position.
Vector<T> TokenProbabilities { get; set; } Gets the sampled token probabilities (p(token) for each drafted token).
Vector<int> Tokens { get; set; } Gets the generated draft tokens.
int NumTokens { get; } Gets the number of draft tokens generated.