CrossModalConsistencyChecker<T>
Models & Types · AiDotNet.Safety.Multimodal
Checks consistency between different modalities (text, image, audio) to detect misaligned or manipulated multimodal content.
Implements: ITextSafetyModule<T>
For Beginners
When you have content with both text and images (or audio), sometimes the text says one thing but the image shows something different or harmful. This module catches those mismatches — for example, a message that says “cute animals” but contains a violent image.
How It Works
Cross-modal attacks exploit the gap between modalities — for example, a benign text description paired with a harmful image, or a safe-looking image with hidden toxic text. This module detects such misalignment by comparing safety signals across modalities.
Detection approach:
- Run each modality through its respective safety module independently
- Compare finding severity across modalities
- Flag cases where one modality is safe but another is unsafe (potential evasion)
- Flag cases where text-image semantic similarity is very low (potential mismatch)
References:
- OmniSafeBench-MM: 9 risk domains, 50 fine-grained categories (2025)
- MM-SafetyBench: 13 scenarios for multimodal safety (ECCV 2024)
- Cross-modal jailbreak attacks on multimodal LLMs (2024)
- AnyAttack: Transferable adversarial attacks on vision-language models (2024)
Constructors
| Constructor | Summary |
|---|---|
CrossModalConsistencyChecker(double mismatchThreshold) |
Initializes a new cross-modal consistency checker. |
Properties
| Property | Summary |
|---|---|
bool IsReady { get; } |
|
string ModuleName { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<SafetyFinding> CheckConsistency(IReadOnlyList<SafetyFinding> textFindings, IReadOnlyList<SafetyFinding> imageFindings, IReadOnlyList<SafetyFinding> audioFindings) |
Compares safety findings from multiple modalities and returns cross-modal findings. |
IReadOnlyList<SafetyFinding> Evaluate(Vector<T> content) |
|
IReadOnlyList<SafetyFinding> EvaluateText(string text) |
Evaluates text for cross-modal consistency indicators. |