IMultimodalSafetyModule<T>
Interfaces · AiDotNet.Safety.Multimodal
Interface for multimodal safety modules that analyze cross-modal content interactions.
Implements: ISafetyModule<T>
Known implementations: MultimodalSafetyModuleBase<T>
For Beginners
A multimodal safety module checks the combination of different content types together. For example, someone might pair innocent-sounding text with a harmful image to bypass safety checks — this module catches those cross-modal attacks.
How It Works
Multimodal safety modules detect risks that arise from the interaction between different content modalities — for example, safe text paired with an unsafe image, or cross-modal attacks that exploit mismatches between text and image safety classifiers.
References:
- Cross-modal safety mechanism transfer failure in VLMs (2024, arxiv:2410.12662)
- OmniSafeBench-MM: 13 attacks, 15 defenses, 9 risk domains (2025, arxiv:2512.06589)
- MM-SafetyBench: 5,040 text-image pairs across 13 scenarios (ECCV 2024)
- Llama Guard 3 Vision: Multimodal safety classification (Meta, 2024, arxiv:2411.10414)
Methods
| Method | Summary |
|---|---|
IReadOnlyList<SafetyFinding> EvaluateTextAudio(string text, Vector<T> audio, int sampleRate) |
Evaluates a text-audio pair for cross-modal safety risks. |
IReadOnlyList<SafetyFinding> EvaluateTextImage(string text, Tensor<T> image) |
Evaluates a text-image pair for cross-modal safety risks. |