InputOutputProgramFitnessEvaluator
Models & Types · AiDotNet.Evolution.Programs
Scores a candidate program by running it on input/output examples through a caller-supplied engine.
Implements: IProgramFitnessEvaluator
For Beginners
This evaluator checks a generated program the way a teacher marks homework: it runs
the program on each example input and compares what it printed with the expected answer, then reports the
fraction it got right. You supply the “runner” that actually executes the code, because running code a language
model wrote is a security decision only you can make — a container or an isolated process is the usual choice.
Pick a ProgramOutputComparison that matches how precise the expected output has to be.
How It Works
Quality is the fraction of ProgramInputOutputExample cases whose captured output matches the
expected output, so it is always between zero and one and always comparable between candidates. Execution goes
through the IProgramExecutionEngine the caller supplies, which is the whole point: generated
program text is untrusted, the core library never runs it in the AiDotNet process, and the sandbox, container,
or remote runner that does run it stays entirely under the caller’s control. A candidate that crashes, times
out, or prints the wrong answer simply scores lower; it never stops the run.
Failure text from the engine is truncated and stripped of control characters before it reaches a diagnostic, so
a program that prints a megabyte to standard error cannot bloat a checkpoint or leak raw payloads into a log.
At most eight example-level diagnostics are retained. VersionHash incorporates a hash of the
examples and the comparison mode, so changing the test set correctly invalidates older checkpoints.
Constructors
| Constructor | Summary |
|---|---|
InputOutputProgramFitnessEvaluator(IProgramExecutionEngine engine, IEnumerable<ProgramInputOutputExample> examples, ProgramOutputComparison comparison, string id) |
Initializes an input/output evaluator. |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<ProgramInputOutputExample> Examples { get; } |
Gets the input/output cases this evaluator scores against. |
ProgramOutputComparison Comparison { get; } |
Gets how captured output is compared with expected output. |
string Id { get; } |
|
string VersionHash { get; } |
Methods
| Method | Summary |
|---|---|
ValueTask<EvolutionTaskResult> EvaluateAsync(ProgramGenome candidate, EvolutionEvaluationContext context, CancellationToken cancellationToken) |