ComprehensiveSafetyBenchmark<T>
Models & Types ยท AiDotNet.Safety.Benchmarking
Comprehensive safety benchmark that aggregates all individual benchmarks into a single suite.
Inheritance: SafetyBenchmarkBase<T> โ ComprehensiveSafetyBenchmark<T>
For Beginners
Instead of running each safety benchmark separately, this runs all of them at once and gives you a complete picture of how well your safety system works across all categories. Use this for a one-stop safety evaluation.
How It Works
Runs all available safety benchmarks (toxicity, jailbreak, bias, PII, hallucination, watermark, adversarial) and produces a unified report with per-category breakdowns. This is the recommended benchmark to run for a complete safety evaluation of your pipeline.
References:
- SafetyBench: Multi-category safety evaluation (ACL 2024)
- HarmBench: Standardized evaluation across attack types (ICML 2024)
- WildGuardTest: 1.7k labeled examples across 13 risk categories (Allen AI, 2024)
Constructors
| Constructor | Summary |
|---|---|
ComprehensiveSafetyBenchmark() |
Initializes a new comprehensive safety benchmark with all default sub-benchmarks. |
ComprehensiveSafetyBenchmark(IReadOnlyList<ISafetyBenchmark<T>> benchmarks) |
Initializes a new comprehensive safety benchmark with specific sub-benchmarks. |
Properties
| Property | Summary |
|---|---|
string BenchmarkName { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyDictionary<string, SafetyBenchmarkResult> RunAllBenchmarks(SafetyPipeline<T> pipeline) |
Runs all sub-benchmarks and returns individual results keyed by benchmark name. |
SafetyBenchmarkResult RunBenchmark(SafetyPipeline<T> pipeline) |