Safety

195 public types in this namespace, organized by kind.

Models & Types (108)

Type Summary
AcousticToxicityDetector<T> Detects toxic/aggressive speech patterns directly from acoustic features without transcription.
AdversarialBenchmark<T> Benchmark for evaluating adversarial robustness of the safety pipeline against evasion attacks.
AdversarialImageEvaluator<T> Detects adversarial perturbations in images via the learnable feature-squeezing ensemble described by Xu et al.
AdversarialRobustnessEvaluator<T> Evaluates text inputs for adversarial perturbations designed to evade safety filters.
AudioSafetyResult Detailed result from audio safety evaluation.
AudioSealWatermarker<T> Localized keyed audio watermark: the spread-spectrum mark of AudioWatermarker, detected segment by segment so it reports which parts of a recording carry the mark.
AudioWatermarker<T> Keyed spread-spectrum audio watermark: a key-derived +/-1 sequence added to the 1-7 kHz block-DCT coefficients at a level that follows each frame’s loudness, detected by correlation.
BiasBenchmark<T> Benchmark for evaluating bias and fairness detection accuracy across demographic groups.
BiasReport Detailed report from bias and fairness evaluation.
ClaimVerdict Verdict for a single claim in a hallucination check.
ClassifierToxicityDetector<T> Detects toxic text using a trained linear classifier over character n-gram features.
ComplianceReport Detailed report from regulatory compliance evaluation.
ComplianceViolation A specific compliance violation found during evaluation.
CompositeGuardrail<T> Chains multiple guardrails into a single composite guardrail that runs all child guardrails in sequence and aggregates their findings.
CompositePIIDetector<T> Combines multiple PII detection strategies into a unified detector with deduplication.
ComprehensiveSafetyBenchmark<T> Comprehensive safety benchmark that aggregates all individual benchmarks into a single suite.
ConsistencyDeepfakeDetector<T> Detects deepfake/AI-generated images by checking spatial consistency and natural image statistics violations.
ContextAwarePIIDetector<T> Context-aware PII detector that reduces false positives by analyzing surrounding text.
CopyrightMatch A match between generated text and potentially copyrighted content.
CopyrightResult Detailed result from copyright and memorization detection.
CrossModalConsistencyChecker<T> Checks consistency between different modalities (text, image, audio) to detect misaligned or manipulated multimodal content.
CustomRule A custom safety rule that evaluates text and optionally returns a finding.
CustomRuleGuardrail<T> User-defined guardrail that applies custom validation rules to text content.
DeepfakeResult Detailed result from deepfake and AI-generated image detection.
DemographicParityChecker<T> Checks for demographic parity violations by detecting differential treatment of demographic groups in model outputs.
EUAIActComplianceChecker<T> Checks compliance with the EU AI Act requirements for AI systems.
EmbeddingCopyrightDetector<T> Detects potential copyright violations using embedding-based semantic similarity to known copyrighted works.
EmbeddingToxicityDetector<T> Detects toxic text using embedding-based cosine similarity to known toxic concept vectors.
EnsembleImageSafetyClassifier<T> Combines multiple image safety classifiers into a weighted ensemble for robust detection.
EnsembleJailbreakDetector<T> Combines multiple jailbreak detection strategies into a robust ensemble.
EnsembleToxicityDetector<T> Combines multiple toxicity detectors into a weighted ensemble for improved accuracy.
EntailmentHallucinationDetector<T> Detects hallucinations using textual entailment (NLI) principles: checking whether reference documents entail (support) each claim in the model output.
EqualizedOddsChecker<T> Checks for equalized odds violations by analyzing whether model quality or effort varies across demographic groups mentioned in the text.
FrameSamplingVideoModerator<T> Video content moderator built on Temporal Segment Networks: sparse segment-wise sampling with a segmental consensus over snippet scores, producing one video-level verdict per category.
FrequencyDeepfakeDetector<T> Detects AI-generated/deepfake images by analyzing frequency domain artifacts.
FrequencyImageWatermarker<T> Keyed Fourier-ring image watermarker: adds a key-derived pattern to a ring-shaped (annular) band of the image’s Fourier spectrum and detects it by correlation.
GDPRComplianceChecker<T> Checks compliance with GDPR requirements related to AI and personal data processing.
GradientJailbreakDetector<T> Detects gradient-based adversarial jailbreak attacks by analyzing token-level anomalies in the input text.
GroupBiasResult Bias analysis result for a specific demographic group.
GuardrailResult Result from guardrail evaluation.
GuardrailRule Defines a declarative guardrail rule with a condition expression and corresponding action.
HallucinationBenchmark<T> Benchmark for evaluating hallucination and factual grounding detection accuracy.
HallucinationResult Detailed result from hallucination detection with per-claim verdicts.
ImageSafetyResult Detailed result from image safety classification.
ImageWatermarker<T> The safety pipeline’s image watermarker: the keyed spread-spectrum DCT watermark of InvisibleImageWatermarker with a configurable reporting threshold.
InputGuardrail<T> Input guardrail that validates user input before it reaches the model.
IntersectionalBiasDetector<T> Detects intersectional bias — bias that uniquely affects individuals at the intersection of multiple demographic identities (e.g., Black women, elderly Asian men).
InvisibleImageWatermarker<T> Keyed spread-spectrum image watermarker that hides a pseudo-random +/-1 sequence in the image’s mid- and high-frequency block DCT coefficients and detects it by correlation.
JailbreakBenchmark<T> Benchmark for evaluating jailbreak and prompt injection detection accuracy.
KnowledgeTripletHallucinationDetector<T> Detects hallucinations by extracting (subject, predicate, object) knowledge triplets and verifying them against reference documents.
LexicalWatermarker<T> Keyed synonym-substitution text watermark: at every word from a synonym group, the key and the preceding word choose which synonym the text uses.
MaskingVoiceProtector<T> Protects voice recordings against cloning using psychoacoustic masking — adding noise that is hidden beneath audible content but disrupts speaker embedding extraction.
MultimodalGuardrail<T> Unified guardrail for vision-language models (VLMs) and multimodal AI systems that validates both text and image content together.
MultimodalSafetyResult Detailed result from multimodal safety evaluation.
MultimodalVideoModerator<T> Comprehensive video moderator that combines frame-level content classification, temporal deepfake detection, and optional audio track analysis.
NERPIIDetector<T> Detects PII using Named Entity Recognition (NER) heuristics based on contextual patterns.
NgramCopyrightDetector<T> Detects potential copyright violations by measuring n-gram overlap with known copyrighted works.
OutputGuardrail<T> Output guardrail that validates model output before it reaches the user.
PIIBenchmark<T> Benchmark for evaluating PII (Personally Identifiable Information) detection accuracy.
PIIEntity Represents a detected PII entity in text.
PIIResult Detailed result from PII detection with detected entities and redacted text.
PatternJailbreakDetector<T> Pattern-based jailbreak and prompt injection detector.
PerplexityMemorizationDetector<T> Detects potential training data memorization by estimating text perplexity.
PerturbationVoiceProtector<T> Protects voice recordings against cloning by adding imperceptible adversarial perturbations.
ProvenanceDeepfakeDetector<T> Detects deepfake/AI-generated images by analyzing provenance signals: compression artifacts, statistical fingerprints, and embedded watermark traces.
ReferenceBasedHallucinationDetector<T> Detects hallucinations by comparing model output against provided reference documents.
RegexPIIDetector<T> Regex-based PII (Personally Identifiable Information) detector for common PII patterns.
RepresentationalBiasDetector<T> Detects representational bias by analyzing whether demographic groups are underrepresented, overrepresented, or systematically associated with specific roles/contexts in text.
RuleBasedToxicityDetector<T> Rule-based toxicity detector using pattern matching for harmful content detection.
SOC2ComplianceChecker<T> Checks compliance with SOC 2 requirements for AI system security and availability.
SafetyBenchmarkCase A single test case for the safety benchmark.
SafetyBenchmarkCategoryResult Per-category benchmark results.
SafetyBenchmarkReport Comprehensive report from running all safety benchmarks.
SafetyBenchmarkResult Results from running a safety benchmark.
SafetyBenchmarkRunner<T> Runs safety benchmarks against a configured safety pipeline to measure detection performance.
SafetyFinding Represents a single safety finding from a safety module evaluation.
SafetyPipeline<T> Composable safety pipeline that runs multiple safety modules and aggregates their findings.
SafetyReport Unified safety report aggregating findings from all safety modules in the pipeline.
SamplingWatermarker<T> The green-list language-model watermark of Kirchenbauer et al.
SceneGraphSafetyClassifier<T> Scene graph-based image safety classifier that analyzes spatial relationships between detected entities to identify unsafe content configurations.
SelfConsistencyHallucinationDetector<T> Detects hallucinations by checking internal consistency of claims within the text.
SemanticJailbreakDetector<T> Detects jailbreak attempts using semantic embedding similarity to known attack patterns.
SpectralAudioWatermarker<T> Keyed patchwork audio watermark: the key splits each frame’s 0.5-6 kHz block-DCT coefficients into two halves, raises the magnitudes of one and lowers those of the other, and detection tests the log-magnitude difference.
StegaStampWatermarker<T> StegaStamp: a learned image watermark.
StereotypeDetector<T> Detects stereotypical associations between demographic groups and attributes in text.
SyntacticWatermarker<T> Keyed form-transformation text watermark: at every negation or pronoun-auxiliary pair that can be written either in full or contracted (“do not” / “don’t”, “they are” / “they’re”), the key and the preceding word choose the form.
SynthIdImageWatermarker<T> A SynthID-Image-style post-hoc watermark.
TemporalConsistencyDetector<T> Detects deepfake videos by analyzing temporal consistency between consecutive frames.
TextImageAlignmentChecker<T> Checks whether an image matches its caption, to catch unsafe images passed off under a safe description.
TextWatermarker<T> The safety pipeline’s text watermarker: the green-list watermark of SamplingWatermarker (Kirchenbauer et al.
TopicRestrictionGuardrail<T> Guardrail that restricts conversation to approved topics using keyword and pattern matching.
ToxicSpan A span of text identified as toxic.
ToxicityBenchmark<T> Benchmark for evaluating toxicity detection accuracy across hate speech, threats, and harassment.
ToxicityResult Detailed result from toxicity detection with per-category scores and spans.
TranscriptionToxicityDetector<T> Detects toxic content in audio by analyzing acoustic features indicative of aggressive speech.
TreeRingWatermarker<T> Tree-Ring watermarking for diffusion models: a keyed ring pattern written into the Fourier transform of the initial noise, which the sampler then turns into an image, detected by inverting the sampler back to noise.
ViTImageSafetyClassifier<T> Patch-statistics image safety classifier: hand-set colour and edge rules evaluated per image patch and pooled with fixed similarity weights.
VideoSafetyResult Detailed result from video safety evaluation with per-frame annotations.
VoiceprintDeepfakeDetector<T> Detects deepfake audio by analyzing speaker voiceprint consistency throughout the recording.
WatermarkBenchmark<T> Benchmark for evaluating watermark detection accuracy across text watermarking techniques.
WatermarkDecision A calibrated three-state decision on one image, with the evidence behind it.
WatermarkDeepfakeDetector<T> Detects AI-generated audio by looking for the presence or absence of known watermark patterns (e.g., AudioSeal-style localized watermarks).
WatermarkDetector<T> Composite text watermark detector: runs the green-list, synonym and contraction detectors of one key and reports the strongest evidence, corrected for having run three tests.
WatermarkKey The secret key of a keyed watermark, and the pseudo-random values every keyed watermarker derives from it.
WatermarkResult Detailed result from watermark detection across any modality.
WatermarkVoiceProtector<T> Protects voice recordings by embedding imperceptible watermarks that survive voice cloning and can be detected in cloned output.
WatermarkedCausalLanguageModel<T> A language model whose next-token logits carry a generation-time watermark: the inner model’s logits plus the watermarker’s green-list bias for the current context.
WatermarkedIncrementalCausalLanguageModel<T> The incremental (KV-cached) form of WatermarkedCausalLanguageModel: it tracks the context the inner model has seen, so each appended token’s logits are biased for the right green list.

Base Classes (22)

Type Summary
AudioDeepfakeDetectorBase<T> Abstract base class for audio deepfake detection modules.
AudioSafetyModuleBase<T> Abstract base class for audio safety modules.
AudioWatermarkerBase<T> Abstract base class for keyed audio watermarkers that mark the block-DCT spectrum of the audio with a pattern derived from a secret key, and detect it by correlation.
ComplianceModuleBase<T> Abstract base class for regulatory compliance checking modules.
CopyrightDetectorBase<T> Abstract base class for copyright and memorization detection modules.
DeepfakeDetectorBase<T> Abstract base class for image deepfake detection modules.
GuardrailBase<T> Abstract base class for guardrail modules.
HallucinationDetectorBase<T> Abstract base class for hallucination detection modules.
ImageSafetyClassifierBase<T> Abstract base class for image safety classifiers.
ImageSafetyModuleBase<T> Abstract base class for image safety modules.
ImageWatermarkerBase<T> Abstract base class for keyed image watermarkers: each adds a pseudo-random pattern derived from a secret key to a transform of the image, and detects it by correlating the same transform against the same pattern.
JailbreakDetectorBase<T> Abstract base class for jailbreak and prompt injection detection modules.
KeyedSubstitutionWatermarker<T> Base class for post-hoc text watermarks that rewrite substitution sites to a keyed choice: at every occurrence of a phrase from a group of interchangeable phrases, the key and the preceding word select which member of the group the text use…
MultimodalSafetyModuleBase<T> Abstract base class for multimodal safety modules that analyze cross-modal content.
PIIDetectorBase<T> Abstract base class for PII detection modules.
SafetyBenchmarkBase<T> Abstract base class for safety benchmark modules.
SafetyModuleBase<T> Abstract base class for all safety modules, providing common infrastructure.
TextSafetyModuleBase<T> Abstract base class for text safety modules providing common text processing utilities.
TextToxicityDetectorBase<T> Abstract base class for toxicity detection modules.
TextWatermarkerBase<T> Abstract base class for keyed text watermarkers.
VideoSafetyModuleBase<T> Abstract base class for video safety modules.
VoiceProtectorBase<T> Abstract base class for voice protection modules.

Interfaces (20)

Type Summary
IAudioDeepfakeDetector<T> Interface for audio deepfake and voice cloning detection modules.
IAudioToxicityDetector<T> Interface for audio toxicity detection modules that identify harmful speech content.
IAudioWatermarker<T> Interface for keyed audio watermarkers that embed an inaudible watermark in audio and detect it later.
IComplianceModule<T> Interface for regulatory compliance checking modules.
ICopyrightDetector<T> Interface for copyright and memorization detection modules.
IDeepfakeDetector<T> Interface for deepfake and AI-generated image detection modules.
IGenerationWatermarker<T> A text watermarker that embeds its keyed watermark while a language model generates, by biasing the model’s next-token logits.
IGuardrail<T> Interface for guardrail modules that validate and filter input/output content.
IHallucinationDetector<T> Interface for hallucination detection modules that identify fabricated or unfaithful content.
IImageSafetyClassifier<T> Interface for image safety classifiers that detect NSFW, violent, or otherwise harmful images.
IImageWatermarker<T> Interface for keyed image watermarkers that embed an imperceptible watermark in an image and detect it later.
IJailbreakDetector<T> Interface for jailbreak and prompt injection detection modules.
IKeyedWatermarker A watermarker whose mark is derived from a WatermarkKey.
IMultimodalSafetyModule<T> Interface for multimodal safety modules that analyze cross-modal content interactions.
IPIIDetector<T> Interface for PII (Personally Identifiable Information) detection modules.
ISafetyBenchmark<T> Interface for safety benchmark modules that evaluate safety module accuracy.
ITextToxicityDetector<T> Interface for toxicity detection modules that identify harmful, abusive, or toxic text content.
ITextWatermarkEmbedder<T> A text watermarker that embeds its keyed watermark into finished text.
ITextWatermarker<T> Interface for keyed text watermarkers that detect, and embed, a watermark in text.
IVoiceProtector<T> Interface for voice protection modules that defend against voice cloning and deepfake attacks.

Enums (12)

Type Summary
AudioWatermarkType The type of audio watermarking technique to use.
GuardrailConditionType Specifies the type of condition a guardrail rule checks.
GuardrailDirection Specifies whether a guardrail checks input, output, or both directions.
HarmCategory The six-category taxonomy of online harm for video platforms.
ImageClassifierType The type of image safety classifier to use.
ImageWatermarkType The type of image watermarking technique to use.
RiskLevel EU AI Act risk classification levels (Articles 5, 6, 50, 52).
SegmentalConsensus The segmental consensus function G of a Temporal Segment Network, which aggregates per-snippet class scores into one video-level score.
TextWatermarkType The type of text watermarking technique to use.
TreeRingKeyPattern The key written into a TreeRingWatermarker’s Fourier mask (Wen et al.
VoiceProtectionType The type of voice protection technique to use.
WatermarkVerdict The outcome of a calibrated watermark decision.

Options & Configuration (29)

Type Summary
AdversarialImageEvaluatorOptions Configuration options for AdversarialImageEvaluator.
AudioSafetyConfig Configuration for audio safety detection modules.
AudioSafetyConfig Configuration for audio safety modules.
AudioWatermarkConfig Configuration for audio watermarking modules.
BiasConfig Configuration for bias and fairness detection modules.
ComplianceConfig Configuration for regulatory compliance checking modules.
ComplianceConfig Configuration for regulatory compliance.
CopyrightDetectorConfig Configuration for copyright and memorization detection modules.
DeepfakeDetectorConfig Configuration for deepfake and AI-generated image detection modules.
FairnessConfig Configuration for fairness and bias detection.
GuardrailConfig Configuration for input/output guardrails.
GuardrailConfig Configuration for guardrail modules.
HallucinationDetectorConfig Configuration for hallucination detection modules.
ImageSafetyConfig Configuration for image safety classification modules.
ImageSafetyConfig Configuration for image safety modules.
ImageWatermarkConfig Configuration for image watermarking modules.
JailbreakDetectorConfig Configuration for jailbreak and prompt injection detection modules.
MultimodalSafetyConfig Configuration for multimodal safety modules.
PIIDetectorConfig Configuration for PII detection modules.
SafetyConfig Master configuration for the comprehensive safety pipeline.
StegaStampOptions Configuration of a StegaStampWatermarker: the image and secret sizes and the training schedule of Tancik et al.
SynthIdImageOptions Configuration of a SynthIdImageWatermarker.
TextSafetyConfig Configuration for text safety modules.
TextWatermarkConfig Configuration for text watermarking modules.
ToxicityDetectorConfig Configuration for toxicity detection modules.
VideoSafetyConfig Configuration for video safety detection modules.
VideoSafetyConfig Configuration for video safety modules.
VoiceProtectorConfig Configuration for voice protection (anti-cloning) modules.
WatermarkConfig Configuration for watermarking modules.

Helpers & Utilities (3)

Type Summary
SafetyPipelineBuilder<T> Fluent builder for constructing a SafetyPipeline with custom modules.
StandardSafetyBenchmarks Provides standard safety benchmark test suites inspired by published safety evaluation datasets.
WatermarkStatistics The detection statistics shared by the keyed watermarkers: z-scores, their one-sided p-values, and the mapping from z to the confidence a watermarker reports.

Exceptions (1)

Type Summary
SafetyViolationException Exception thrown when content fails a safety check and the configuration requires throwing.