IProgramFitnessEvaluator

Interfaces · AiDotNet.Interfaces

Scores one candidate program and reports the result as an evolution task result.

Known implementations: DelegateProgramFitnessEvaluator, InputOutputProgramFitnessEvaluator, LlmJudgeProgramFitnessEvaluator<T>, NoveltyGatingProgramFitnessEvaluator, SandboxedProgramFitnessEvaluator, ScriptProgramFitnessEvaluator

For Beginners

Evolution needs a score for every candidate program, and this interface is where you supply it. A typical implementation writes the program to a temporary file, runs it inside a sandbox with a time limit, compares its output with expected outputs, and returns the fraction that passed. Because the library never runs the code itself, you stay in control of how risky generated code is contained. Return a failed result rather than throwing when a program crashes; a crashing candidate is normal in evolution and should not stop the run.

How It Works

This is the seam through which a program-evolution run learns whether a candidate is any good, and it is deliberately the only place where generated code is executed. The core library never runs untrusted program text in its own process: an implementation hands the source to a sandbox, a container, a separate process, or a remote service of the caller’s choosing, which is why no execution dependency is required to use the rest of the substrate. Implementations must be deterministic for a given EvolutionEvaluationContext, drawing any randomness from CreateRandom, and must report recoverable problems as Failed rather than by throwing so the engine can record the diagnostic and apply its retry policy.

Id names the evaluator and VersionHash changes whenever its semantics or its test data change; ProgramEvolutionTask forwards the latter as its evaluator version hash, so a checkpoint written against one test suite refuses to resume against a different one.

Properties

Property Summary
string Id { get; } Gets a stable evaluator identifier.
string VersionHash { get; } Gets a version hash that changes whenever evaluator semantics or test data change.

Methods

Method Summary
ValueTask<EvolutionTaskResult> EvaluateAsync(ProgramGenome candidate, EvolutionEvaluationContext context, CancellationToken cancellationToken) Scores one candidate program.