LocalEngineChatClient<T>
Models & Types · AiDotNet.Agentic.Models.Local
An IChatClient that runs entirely in-process over an ICausalLanguageModel
— no network, no API key, no external service. It renders the conversation to a prompt, encodes it,
autoregressively samples tokens until the end-of-sequence token or a length limit, and decodes the
result. This is the flagship “local-first” capability: the same agent code that drives OpenAI or
Anthropic drives AiDotNet’s own model.
Implements: IChatClient<T>
For Beginners
This is your own chatbot brain running on your machine. You hand it the conversation; it writes the reply one word-piece at a time until it decides it’s done or hits the length cap. Everything else in this library that talks to a “chat model” can talk to this one instead — so you can build agents with no cloud dependency at all.
How It Works
Because it implements IChatClient, the local engine is a drop-in for every higher layer
(agents, supervisor/swarm, memory). Both non-streaming and streaming generation are supported; streaming
decodes incrementally and yields the new text on each step. Constrained decoding is supported via
Constraint (an ITokenConstraint enforced at the logits, e.g.
AllowedTokenSetConstraint / FiniteStateTokenConstraint). What this slice does
not do: native tool-calling and auto-deriving a constraint from ResponseFormat —
requests that ask for either are rejected with NotSupportedException rather than silently
returning plain text; set Constraint explicitly for guaranteed-structured
output.
Constructors
| Constructor | Summary |
|---|---|
LocalEngineChatClient(ICausalLanguageModel<T> model, IGenerationTokenizer tokenizer, IChatPromptTemplate template, LocalEngineOptions options) |
Initializes a new local engine. |
Properties
| Property | Summary |
|---|---|
string ModelId { get; } |
Methods
| Method | Summary |
|---|---|
IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) |
|
Task<ChatResponse> GetResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) |