BenchmarkingOptions
Options & Configuration · AiDotNet.Configuration
Configuration options for running benchmarks through the AiDotNet facade.
For Beginners
Benchmarking is how you measure model quality using standardized tests. You choose a suite (like GSM8K for math reasoning), and AiDotNet runs it and returns a report.
How It Works
This options class is designed for use with AiModelBuilder.ConfigureBenchmarking(...) and
AiModelResult.EvaluateBenchmarksAsync(...). Users specify which benchmark suites to run,
and AiDotNet orchestrates the execution behind the scenes.
Constructors
| Constructor | Summary |
|---|---|
BenchmarkingOptions() |
Properties
| Property | Summary |
|---|---|
BenchmarkFailurePolicy FailurePolicy { get; set; } |
Gets or sets how failures should be handled. |
BenchmarkReportDetailLevel DetailLevel { get; set; } |
Gets or sets how much detail should be included in reports. |
BenchmarkSuite[] Suites { get; set; } |
Gets or sets the benchmark suites to run. |
CausalLmBenchmarkOptions LanguageModel { get; set; } |
Gets or sets causal language-model benchmark configuration (LAMBADA and Perplexity). |
FederatedTabularBenchmarkOptions Tabular { get; set; } |
Gets or sets federated tabular benchmark configuration (synthetic non-IID suite). |
FederatedTextBenchmarkOptions Text { get; set; } |
Gets or sets federated text benchmark configuration (Sent140/Shakespeare suites). |
FederatedVisionBenchmarkOptions Vision { get; set; } |
Gets or sets federated vision benchmark configuration (FEMNIST/CIFAR suites). |
LeafFederatedBenchmarkOptions Leaf { get; set; } |
Gets or sets LEAF federated benchmark configuration (required when running LEAF). |
bool AttachReportToResult { get; set; } |
Gets or sets whether the generated report should be attached to the model result. |
bool CiMode { get; set; } |
Gets or sets whether benchmarking should run in CI-friendly mode. |
int? SampleSize { get; set; } |
Gets or sets an optional sample size for suites that support sampling. |
int? Seed { get; set; } |
Gets or sets an optional deterministic seed used for CI-friendly sampling. |