WeightedEnsembleChatClient<T>
Models & Types · AiDotNet.Agentic.Pipeline
Routes each chat call to one of several models, chosen by weight from a reproducible random stream.
Implements: IChatClient<T>
For Beginners
This lets you use several AI models as if they were one. Give each model a weight and it gets picked that often; if the picked model errors, the next one is tried automatically. Because the picking is seeded, running your program twice picks the same models in the same order — which is what makes two experiments comparable. You can also ask every model at once when you want several opinions.
How It Works
Mixing a strong model with a cheap one is how a long search stays affordable: the expensive model contributes
the occasional breakthrough and the cheap one contributes volume, in whatever ratio the weights describe. This
client makes that mixture a drop-in IChatClient, so every layer above it — the prompt
builder, the variation operator, the recording pipeline — is unaware there is more than one model.
Three things distinguish it from the reference OpenEvolve ensemble. Selection is drawn from a seeded
StableRandom owned by this client, so it is reproducible and cannot be disturbed by anything
else in the process; upstream seeds Python’s shared module-level generator from the first model’s
configuration and shares it with the rest of the program. A member that fails hands its turn to the remaining
members in weight order instead of failing the iteration; upstream has no fallback at all. And
GetAllResponsesAsync calls every member with bounded concurrency, where upstream’s
equivalent loops sequentially.
Every call is accounted for: GetSelectionCounts reports how often each member was chosen,
GetUsage aggregates the tokens they reported, and a response that arrives without a model id is
stamped with the answering member’s name, so the record always says who actually answered.
Constructors
| Constructor | Summary |
|---|---|
WeightedEnsembleChatClient(IReadOnlyList<ChatClientEnsembleMember<T>> members, WeightedEnsembleChatClientOptions options) |
Initializes a weighted ensemble. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<ChatClientEnsembleMember<T>> Members { get; } |
Gets the members this ensemble routes between, in declaration order. |
long TotalCalls { get; } |
Gets how many calls this ensemble has served. |
string ModelId { get; } |
Methods
| Method | Summary |
|---|---|
ChatUsage GetUsage() |
Gets the token usage reported by every member so far, summed. |
IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) |
|
IReadOnlyDictionary<string, long> GetSelectionCounts() |
Gets how many times each member was selected, keyed by member name. |
Task<ChatResponse> GetResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) |
|
Task<IReadOnlyList<ChatResponse>> GetAllResponsesAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) |
Asks every member the same question, with bounded concurrency, and returns all answers. |
WeightedEnsembleChatClientOptions GetOptions() |
Gets a copy of the settings this ensemble was constructed with. |
Fields
| Field | Summary |
|---|---|
string DefaultModelId |
The model id reported when no other is configured. |