ScriptProgramEvaluationOptions
Options & Configuration · AiDotNet.Configuration
Configures how a caller-supplied evaluator script scores candidate programs inside the sandbox.
For Beginners
Sometimes “is this program any good?” cannot be answered by comparing printed
output — you want to score style, structure, or a domain-specific rule. This lets you write that judgement as a
small script. The script receives the candidate’s source code on its standard input and prints one JSON object
such as {"quality": 0.8} back. These settings say what that script is, what language it is in, and how
much of its extra commentary to keep.
How It Works
The evaluator script is itself untrusted code — it is usually written by the same model that writes the
candidates — so it runs through exactly the same execution engine, under exactly the same time, memory, output,
and concurrency limits. The reference OpenEvolve implementation writes its evaluator to
evaluator_<hash>.py and executes it unsandboxed inside its worker processes; running an evaluator
with fewer protections than the thing it evaluates is a hole this option set closes by construction, because
there is nowhere here to specify a second, laxer sandbox.
EvaluatorScript and EvaluatorScriptLanguage hold the script and the language it is
written in. RequireEntryPoint checks for EntryPointMarker when the evaluator is
constructed rather than when the first candidate is scored, so a script missing its entry point is a
configuration error you see immediately instead of a run that fails thousands of evaluations later.
Constructors
| Constructor | Summary |
|---|---|
ScriptProgramEvaluationOptions() |
Properties
| Property | Summary |
|---|---|
EvolutionOptimizationDirection Direction { get; set; } |
Gets or sets whether a larger reported quality is better. |
ProgramLanguage EvaluatorScriptLanguage { get; set; } |
Gets or sets the language the evaluator script is written in. |
bool RequireEntryPoint { get; set; } |
Gets or sets whether EntryPointMarker is enforced when the evaluator is constructed. |
int MaxArtifactCount { get; set; } |
Gets or sets how many artifacts from one evaluation become diagnostics. |
int MaxArtifactLength { get; set; } |
Gets or sets how many characters of each artifact are retained. |
string EntryPointMarker { get; set; } |
Gets or sets the text that must appear in the script for it to be accepted. |
string EvaluatorScript { get; set; } |
Gets or sets the evaluator script source, or null when it is supplied directly to the evaluator. |
Methods
| Method | Summary |
|---|---|
ScriptProgramEvaluationOptions Clone() |
Creates an independent copy so a running evaluator is unaffected by later mutation. |
void Validate() |
Rejects a configuration the evaluator could not honour. |
Fields
| Field | Summary |
|---|---|
int MaxArtifactCountCeiling |
The largest number of artifacts retained from one evaluation. |
int MaxArtifactLengthCeiling |
The largest artifact text retained per artifact, in characters. |