LlmProgramVariationOperator<T>

Models & Types · AiDotNet.Evolution.Programs

Proposes the next candidate program by asking a chat model to edit or rewrite the parent.

Implements: ICheckpointableVariationOperator<ProgramGenome>

For Beginners

This is the piece that actually asks an AI to improve your program. It shows the model the current program along with its score, what the tests printed, and a couple of other programs that did well, then reads the answer and hands the improved program back to the search. If the answer cannot be used — a common case, usually because the model asked to replace text that is not in the file — it explains the problem and asks again instead of wasting the round. You supply the chat client, so no model is contacted unless you configure one.

How It Works

The operator turns one archive parent into one new ProgramGenome. It renders the prompt through a ProgramPromptBuilder, so the parent, its score, its archive cell, the sibling programs the selection policy offered as inspiration, and the evaluator’s captured output all reach the model through the templated, bounded, redacted path rather than through ad-hoc string concatenation. The answer is parsed with ProgramDiff or FencedCodeExtractor according to the mode the builder resolved. Nothing is executed here; the resulting genome goes back to the engine, which canonicalizes it and passes it to whatever sandboxed evaluator the caller configured.

Two behaviours make this strictly better than the reference OpenEvolve worker loop. First, an answer whose edits do not apply is not thrown away in silence: the typed failures become a short feedback message and the model is asked again, up to MaxProposalRetries times, where upstream logs “No valid diffs found” and loses the iteration. Second, when every attempt fails the operator returns the parent genome unchanged, so the engine recognizes a duplicate and the proposal costs no evaluator budget at all, whereas upstream a child identical to its parent is evaluated like any other candidate. Every request is recorded as a ProgramProposalAttempt and summed into GetUsage, so the cost of unusable answers is measurable rather than invisible.

Determinism follows the engine contract: the prompt is rendered from the proposal’s StableRandom stream and the sampling seed is drawn from the same stream unless Seed pins one, so replaying a run sends the model identical requests. The prompt builder’s own version hash is folded into VersionHash, so editing a template is visible to checkpoint resume instead of silently changing what the model is shown mid-run. Security follows the untrusted-content rule: program text, artifacts, and diagnostics are truncated and redacted by the builder, feedback messages carry bounded control-character-sanitized excerpts, and a provider exception contributes only its type name, never its message, so credentials and endpoints cannot leak into a prompt or a log.

Supply an IProposalProvenanceSink and every request is additionally written out as a ProposalProvenanceRecord: the parent it started from, the model that answered, a hash of the exact conversation, a bounded and redacted copy of the prompt and the answer, the reported token cost, the measured latency, and what the answer parsed into. Replaying that stream through BuildLineages rebuilds the ancestry of any program the run produced, which is the post-hoc audit and training-data story upstream gets from its per-program files. Recording is entirely opt-in, never blocks a proposal, and a sink failure is counted in ProvenanceFailureCount rather than ending the run. Only requests are recorded: the terminal Exhausted bookkeeping entry is not a request and does not appear in the stream, so every record corresponds to exactly one call.

Constructors

Constructor Summary
LlmProgramVariationOperator(IChatClient<T> chatClient, ProgramEvolutionOptions programOptions, LlmProgramVariationOptions variationOptions, string id, ProgramPromptBuilder promptBuilder, IProposalProvenanceSink provenanceSink, ProposalProvenanceOptions provenanceOptions) Initializes a language-model variation operator.

Properties

Property Summary
ProgramPromptBuilder PromptBuilder { get; } Gets the prompt builder that renders every request this operator sends.
long ProvenanceFailureCount { get; } Gets how many provenance writes failed since this operator was constructed.
string Id { get; }
string VersionHash { get; }

Methods

Method Summary
IReadOnlyList<ProgramProposalAttempt> GetRecentAttempts() Gets the most recent recorded proposal attempts, oldest first.
LlmProgramVariationOptions GetVariationOptions() Gets a copy of the prompting and retry settings this operator uses.
ProgramEvolutionLlmUsage GetUsage() Gets the language-model totals accumulated since this operator was constructed.
ProgramEvolutionOptions GetProgramOptions() Gets a copy of the program bounds this operator enforces on every proposal.
ValueTask<ProgramGenome> ProposeAsync(EvolutionVariationContext<ProgramGenome> context, CancellationToken cancellationToken)
string CaptureState()
void RestoreState(string state)