NgramCopyrightDetector<T>
Models & Types · AiDotNet.Safety.Text
Detects potential copyright violations by measuring n-gram overlap with known copyrighted works.
Inheritance: ModelBase<T, Vector<T>, Vector<T>> → SafetyModuleBase<T> → TextSafetyModuleBase<T> → NgramCopyrightDetector<T>
For Beginners
If an AI reproduces large chunks of a book, article, or other copyrighted work word-for-word, that’s a copyright problem. This module checks how much of the output matches known copyrighted texts by comparing sequences of words.
How It Works
Computes word-level n-gram overlap between the model output and a corpus of known copyrighted texts. High overlap (long verbatim sequences) indicates potential memorization or copyright infringement. The detector uses sliding window analysis to identify the longest matching subsequences.
Detection approach:
- Extract word n-grams (n=4,5,6) from the output
- Check against indexed n-grams from known copyrighted works
- Compute overlap ratio — high overlap indicates potential infringement
- Identify longest matching subsequence as evidence
References:
- DE-COP: Detecting copyrighted content via paraphrased permutations (2024, arxiv:2402.09910)
- Copyright pre-training data filtering (2025, arxiv:2512.02047)
- Machine unlearning to remove memorized copyrighted content (2024, arxiv:2412.18621)
Constructors
| Constructor | Summary |
|---|---|
NgramCopyrightDetector(string[] copyrightedTexts, string[] sourceNames, double threshold, int minNgramLength) |
Initializes a new n-gram copyright detector. |
Properties
| Property | Summary |
|---|---|
string ModuleName { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<SafetyFinding> EvaluateText(string text) |