WeightedEnsembleChatClient<T>

Models & Types · AiDotNet.Agentic.Pipeline

Routes each chat call to one of several models, chosen by weight from a reproducible random stream.

Implements: IChatClient<T>

For Beginners

This lets you use several AI models as if they were one. Give each model a weight and it gets picked that often; if the picked model errors, the next one is tried automatically. Because the picking is seeded, running your program twice picks the same models in the same order — which is what makes two experiments comparable. You can also ask every model at once when you want several opinions.

How It Works

Mixing a strong model with a cheap one is how a long search stays affordable: the expensive model contributes the occasional breakthrough and the cheap one contributes volume, in whatever ratio the weights describe. This client makes that mixture a drop-in IChatClient, so every layer above it — the prompt builder, the variation operator, the recording pipeline — is unaware there is more than one model.

Three things distinguish it from the reference OpenEvolve ensemble. Selection is drawn from a seeded StableRandom owned by this client, so it is reproducible and cannot be disturbed by anything else in the process; upstream seeds Python’s shared module-level generator from the first model’s configuration and shares it with the rest of the program. A member that fails hands its turn to the remaining members in weight order instead of failing the iteration; upstream has no fallback at all. And GetAllResponsesAsync calls every member with bounded concurrency, where upstream’s equivalent loops sequentially.

Every call is accounted for: GetSelectionCounts reports how often each member was chosen, GetUsage aggregates the tokens they reported, and a response that arrives without a model id is stamped with the answering member’s name, so the record always says who actually answered.

Constructors

Constructor Summary
WeightedEnsembleChatClient(IReadOnlyList<ChatClientEnsembleMember<T>> members, WeightedEnsembleChatClientOptions options) Initializes a weighted ensemble.

Properties

Property Summary
IReadOnlyList<ChatClientEnsembleMember<T>> Members { get; } Gets the members this ensemble routes between, in declaration order.
long TotalCalls { get; } Gets how many calls this ensemble has served.
string ModelId { get; }

Methods

Method Summary
ChatUsage GetUsage() Gets the token usage reported by every member so far, summed.
IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken)
IReadOnlyDictionary<string, long> GetSelectionCounts() Gets how many times each member was selected, keyed by member name.
Task<ChatResponse> GetResponseAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken)
Task<IReadOnlyList<ChatResponse>> GetAllResponsesAsync(IReadOnlyList<ChatMessage> messages, ChatOptions options, CancellationToken cancellationToken) Asks every member the same question, with bounded concurrency, and returns all answers.
WeightedEnsembleChatClientOptions GetOptions() Gets a copy of the settings this ensemble was constructed with.

Fields

Field Summary
string DefaultModelId The model id reported when no other is configured.