LlmProgramNoveltyJudge<T>
Models & Types · AiDotNet.Evolution.Programs.Novelty
Asks a language model whether a candidate is a meaningful change from the incumbent it most resembles.
Implements: IProgramNoveltyJudge
For Beginners
This is the expensive last resort of a novelty check. Two programs look nearly identical to the cheap checks, so a language model is shown both and asked whether the change is real. If it cannot answer, this says so rather than guessing, and your policy decides what to do.
How It Works
The prompt asks for the same distinction the reference implementation asks for — algorithmic, structural, functional, implementation and hyperparameter differences count; renames, formatting and comments do not — but the answer handling differs in three ways that matter.
First, the parse is correct. Upstream instructs the model to answer NOT_NOVEL and then searches the
answer for the string "NOT NOVEL" with a space, which never matches; the underscored form contains
NOVEL at offset four, so upstream reads its own canonical negative answer as a positive one and its judge
can only ever reject when the model spontaneously uses a spacing the prompt never asked for. Here every spelling
— underscore, space, or hyphen — is recognised, and the earliest verdict token in the answer wins so that a
model which reasons first and concludes last is still read correctly.
Second, an unusable answer is reported as Unavailable instead of being
silently converted to “novel”, so the calling policy decides whether an unreachable judge admits or discards.
Third, both programs pass through PromptTextRedactor and a byte budget before they reach the
request, so an evolved program that printed a credential into itself cannot carry it into a provider call, and
a pathological candidate cannot inflate the request without bound.
Constructors
| Constructor | Summary |
|---|---|
LlmProgramNoveltyJudge(IChatClient<T> chatClient, int maxProgramBytes, double? temperature, int? maxOutputTokens, string id) |
Initializes a language-model novelty judge. |
Properties
| Property | Summary |
|---|---|
double? Temperature { get; } |
Gets the sampling temperature used for judging requests, or null for the client’s default. |
int MaxProgramBytes { get; } |
Gets the per-program byte budget applied before a request is sent. |
int? MaxOutputTokens { get; } |
Gets the output token cap used for judging requests, or null for the client’s default. |
long Judgements { get; } |
Gets how many judging requests were sent since this judge was constructed. |
long UnavailableAnswers { get; } |
Gets how many requests produced no usable verdict. |
string Id { get; } |
Methods
| Method | Summary |
|---|---|
ProgramNoveltyVerdict ParseVerdict(string answer) |
Reads a verdict out of a free-text model answer. |
ValueTask<ProgramNoveltyVerdict> JudgeAsync(ProgramGenome candidate, ProgramGenome incumbent, CancellationToken cancellationToken) |
Fields
| Field | Summary |
|---|---|
int DefaultMaxProgramBytes |
The default byte budget applied to each program before it reaches the request. |