CausalLmBenchmarkOptions
Options & Configuration · AiDotNet.Configuration
Configuration options for causal language-model dataset suites (LAMBADA
and Perplexity).
Constructors
| Constructor | Summary |
|---|---|
CausalLmBenchmarkOptions() |
Properties
| Property | Summary |
|---|---|
IReadOnlyList<string> LambadaPassages { get; set; } |
Gets or sets optional in-memory full LAMBADA passages (each ending with the held-out target word). |
ValueTuple<int, int>? CalibrationTokenRange { get; set; } |
Gets or sets an optional calibration/training token range (Start, Count) that is asserted to have strictly zero overlap with the evaluation token slice before scoring. |
int ContextWindowLength { get; set; } |
Gets or sets the non-overlapping context window length (in tokens) for perplexity evaluation (default 512). |
int EvaluationTokenStart { get; set; } |
Gets or sets the starting token index within the tokenized corpus for held-out perplexity evaluation. |
int MaxSubwordTokensPerWord { get; set; } |
Gets or sets the maximum number of subword tokens decoded per word on LAMBADA (default 8). |
int WarmupItems { get; set; } |
Gets or sets the number of untimed warmup items executed prior to recording raw wall-clock latencies (default 1). |
int? EvaluationTokenCount { get; set; } |
Gets or sets an optional token count for the evaluation slice (null evaluates to the end of the corpus). |
string LambadaDatasetPath { get; set; } |
Gets or sets an optional path to a LAMBADA JSONL ({"text": "..."} per line) or plain-text passage file. |
string PerplexityCorpusPath { get; set; } |
Gets or sets an optional path to a plain-text evaluation corpus for Perplexity. |
string PerplexityCorpusText { get; set; } |
Gets or sets optional in-memory evaluation text for Perplexity. |