AiModelResultTransformerExtensions
Helpers & Utilities · AiDotNet.Transformers.Extensions
Model-specific inference extensions on AiModelResult for the
transformer / language-model family. Part of #1836.
How It Works
See AiModelResultRadianceFieldExtensions for the full design rationale — extension
methods live in the same assembly as AiModelResult so they
access internal Model directly without exposing it.
Autoregressive generation is a common transformer inference pattern — the model outputs next-token logits per position, we sample/argmax the last position, append, and repeat. These extensions ship the two most common decoding strategies (greedy argmax + temperature sampling); more sophisticated strategies (beam search, top-k / top-p nucleus) can be layered on the same primitive.
Methods
| Method | Summary |
|---|---|
Task<Tensor<T>> GenerateGreedyAsync(AiModelResult<T, TInput, TOutput> result, Tensor<T> startTokens, int maxNewTokens, int? eosTokenId, IProgress<int> onToken, CancellationToken cancellationToken) |
Async generation with per-token streaming callback. |
Tensor<T> GenerateGreedy(AiModelResult<T, TInput, TOutput> result, Tensor<T> startTokens, int maxNewTokens, int? eosTokenId) |
Runs greedy autoregressive generation — at each step, pick the token with the highest logit. |
Tensor<T> GenerateSampled(AiModelResult<T, TInput, TOutput> result, Tensor<T> startTokens, int maxNewTokens, T temperature, int? eosTokenId, int? seed) |
Runs temperature-sampled autoregressive generation. |
Tensor<T>[] GenerateGreedyBatch(AiModelResult<T, TInput, TOutput> result, Tensor<T>[] startTokensPerPrompt, int maxNewTokens, int? eosTokenId) |
Batched greedy generation — runs N independent prompt sequences in parallel using the .NET thread pool. |