SoftmaxPolicyRouter<T>
Models & Types · AiDotNet.Agentic.SelfImproving
A routing policy over candidate agents, trained by REINFORCE: it keeps a learnable preference weight per
agent (optionally per context), selects via a softmax over those weights, and updates them with a
policy-gradient step from observed rewards. Compared with the simpler mean-reward bandit
(LearnedAgentRouter), this learns a stochastic policy that handles non-stationary rewards
and keeps exploring in proportion to its current confidence.
Implements: IAgent<T>
For Beginners
A dispatcher that keeps a “preference score” for each worker and nudges those scores up or down based on how well the chosen worker did versus the recent average. Over time it sends more work to the workers that do better, while still occasionally trying others.
How It Works
The update for an observed (agent, reward) is the REINFORCE rule with a running-mean baseline:
w[a'] += lr · (reward − baseline) · (1[a'=chosen] − π(a')) for every candidate a'. As a
result, agents that earn above-baseline reward gain probability mass. As an IAgent it
composes with tracing, closing the loop.
Constructors
| Constructor | Summary |
|---|---|
SoftmaxPolicyRouter(IReadOnlyList<IAgent<T>> candidates, double learningRate, Func<IReadOnlyList<ChatMessage>, string> contextKey, int? seed) |
Initializes a new policy router. |
Properties
| Property | Summary |
|---|---|
string Description { get; } |
|
string Name { get; } |
Methods
| Method | Summary |
|---|---|
Task<AgentRunResult> RunAsync(IReadOnlyList<ChatMessage> messages, CancellationToken cancellationToken) |
|
double ProbabilityOf(string agentName, IReadOnlyList<ChatMessage> context) |
Gets the current policy probability of choosing an agent for a context. |
string SelectBestAgentName(IReadOnlyList<ChatMessage> context) |
Returns the highest-probability agent for a context (deterministic, the exploit choice). |
void LearnFrom(IEnumerable<AgentTrajectory> trajectories) |
Trains the policy from graded trajectories (each whose agent is a candidate and whose reward is set). |
void Update(IReadOnlyList<ChatMessage> context, string agentName, double reward) |
Applies a single REINFORCE update for an observed (agent, reward) in the given context. |