CausalLmBenchmarkOptions

Options & Configuration · AiDotNet.Configuration

Configuration options for causal language-model dataset suites (LAMBADA and Perplexity).

Constructors

Constructor Summary
CausalLmBenchmarkOptions()

Properties

Property Summary
IReadOnlyList<string> LambadaPassages { get; set; } Gets or sets optional in-memory full LAMBADA passages (each ending with the held-out target word).
ValueTuple<int, int>? CalibrationTokenRange { get; set; } Gets or sets an optional calibration/training token range (Start, Count) that is asserted to have strictly zero overlap with the evaluation token slice before scoring.
int ContextWindowLength { get; set; } Gets or sets the non-overlapping context window length (in tokens) for perplexity evaluation (default 512).
int EvaluationTokenStart { get; set; } Gets or sets the starting token index within the tokenized corpus for held-out perplexity evaluation.
int MaxSubwordTokensPerWord { get; set; } Gets or sets the maximum number of subword tokens decoded per word on LAMBADA (default 8).
int WarmupItems { get; set; } Gets or sets the number of untimed warmup items executed prior to recording raw wall-clock latencies (default 1).
int? EvaluationTokenCount { get; set; } Gets or sets an optional token count for the evaluation slice (null evaluates to the end of the corpus).
string LambadaDatasetPath { get; set; } Gets or sets an optional path to a LAMBADA JSONL ({"text": "..."} per line) or plain-text passage file.
string PerplexityCorpusPath { get; set; } Gets or sets an optional path to a plain-text evaluation corpus for Perplexity.
string PerplexityCorpusText { get; set; } Gets or sets optional in-memory evaluation text for Perplexity.