ScriptProgramEvaluationOptions

Options & Configuration · AiDotNet.Configuration

Configures how a caller-supplied evaluator script scores candidate programs inside the sandbox.

For Beginners

Sometimes “is this program any good?” cannot be answered by comparing printed output — you want to score style, structure, or a domain-specific rule. This lets you write that judgement as a small script. The script receives the candidate’s source code on its standard input and prints one JSON object such as {"quality": 0.8} back. These settings say what that script is, what language it is in, and how much of its extra commentary to keep.

How It Works

The evaluator script is itself untrusted code — it is usually written by the same model that writes the candidates — so it runs through exactly the same execution engine, under exactly the same time, memory, output, and concurrency limits. The reference OpenEvolve implementation writes its evaluator to evaluator_<hash>.py and executes it unsandboxed inside its worker processes; running an evaluator with fewer protections than the thing it evaluates is a hole this option set closes by construction, because there is nowhere here to specify a second, laxer sandbox.

EvaluatorScript and EvaluatorScriptLanguage hold the script and the language it is written in. RequireEntryPoint checks for EntryPointMarker when the evaluator is constructed rather than when the first candidate is scored, so a script missing its entry point is a configuration error you see immediately instead of a run that fails thousands of evaluations later.

Constructors

Constructor Summary
ScriptProgramEvaluationOptions()

Properties

Property Summary
EvolutionOptimizationDirection Direction { get; set; } Gets or sets whether a larger reported quality is better.
ProgramLanguage EvaluatorScriptLanguage { get; set; } Gets or sets the language the evaluator script is written in.
bool RequireEntryPoint { get; set; } Gets or sets whether EntryPointMarker is enforced when the evaluator is constructed.
int MaxArtifactCount { get; set; } Gets or sets how many artifacts from one evaluation become diagnostics.
int MaxArtifactLength { get; set; } Gets or sets how many characters of each artifact are retained.
string EntryPointMarker { get; set; } Gets or sets the text that must appear in the script for it to be accepted.
string EvaluatorScript { get; set; } Gets or sets the evaluator script source, or null when it is supplied directly to the evaluator.

Methods

Method Summary
ScriptProgramEvaluationOptions Clone() Creates an independent copy so a running evaluator is unaffected by later mutation.
void Validate() Rejects a configuration the evaluator could not honour.

Fields

Field Summary
int MaxArtifactCountCeiling The largest number of artifacts retained from one evaluation.
int MaxArtifactLengthCeiling The largest artifact text retained per artifact, in characters.