MambaCausalLanguageModel<T>
Models & Types · AiDotNet.Agentic.Models.Local
A KV-cached IIncrementalCausalLanguageModel adapter over MambaLanguageModel.
It drives the model’s per-token Step fast path, carrying the recurrent
state (causal-conv window + selective-scan hidden state) so each new token costs O(1) instead of
reprocessing the whole context.
Implements: IIncrementalCausalLanguageModel<T>
For Beginners
A plain forward pass re-reads the entire conversation every time it predicts a word, which gets slower as the text grows. Mamba can instead remember a small summary of everything so far and update it with just the new word — same answer, far less work. This adapter exposes that “remember-as-you-go” ability to the chat engine.
How It Works
The incremental path is mathematically equivalent to the full-sequence forward (Gu & Dao 2023): feeding
tokens one at a time while carrying state reproduces Predict’s logits at
every position. This is verified by unit tests that assert step-by-step output matches the parallel scan.
The base NextTokenLogits still runs a full forward (used when a caller does not opt into the
cache); the fast path is StartSequence + AppendToken.
Constructors
| Constructor | Summary |
|---|---|
MambaCausalLanguageModel(MambaLanguageModel<T> network, int vocabularySize) |
Initializes a new KV-cached adapter over a Mamba language model. |
Properties
| Property | Summary |
|---|---|
int VocabularySize { get; } |
Methods
| Method | Summary |
|---|---|
Vector<T> AppendToken(int tokenId) |
|
Vector<T> NextTokenLogits(IReadOnlyList<int> tokenIds) |
|
Vector<T> StartSequence(IReadOnlyList<int> promptTokenIds) |
|
void ResetCache() |