WatermarkBenchmark<T>

Models & Types · AiDotNet.Safety.Benchmarking

Benchmark for evaluating watermark detection accuracy across text watermarking techniques.

Inheritance: SafetyBenchmarkBase<T> → WatermarkBenchmark<T>

For Beginners

Watermarks are hidden signatures in AI-generated text that help identify it as AI-generated. This benchmark tests whether your system can detect various types of watermarks without false-flagging normal text.

How It Works

Measures false positives: none of these texts carries a watermark under the detector’s key. The detectors are keyed (Kirchenbauer et al. 2023), so text that merely looks watermarked, with an unusual vocabulary or systematic synonym choices, is natural text to them and must not be flagged. Detection itself needs text embedded with the key; the keyed round-trip tests cover it.

References:

  • SynthID-Text: Watermarking for LLMs (Google DeepMind, 2024)
  • A Watermark for Large Language Models (Kirchenbauer et al., ICML 2023)
  • Unforgeable watermarking for language models (Christ et al., 2024)

Constructors

Constructor Summary
WatermarkBenchmark()

Properties

Property Summary
string BenchmarkName { get; }

Methods

Method Summary
SafetyBenchmarkResult RunBenchmark(SafetyPipeline<T> pipeline)