CharNGramVectorizer<T>
Models & Types · AiDotNet.Preprocessing.TextVectorizers
Converts text documents to character-level n-gram feature vectors.
Inheritance: TextVectorizerBase<T> → CharNGramVectorizer<T>
For Beginners
Character n-grams capture patterns at the letter level:
- “hello” with (2,3) n-grams produces: “he”, “el”, “ll”, “lo”, “hel”, “ell”, “llo”
- Robust to spelling mistakes (“color” and “colour” share many character n-grams)
- Works well for author identification and language detection
- Can capture morphological patterns (prefixes, suffixes)
How It Works
CharNGramVectorizer creates features from character sequences rather than words. This is particularly useful for:
- Handling misspellings and typos
- Languages without clear word boundaries
- Capturing subword patterns
- Short text classification (tweets, SMS)
Constructors
| Constructor | Summary |
|---|---|
CharNGramVectorizer(ValueTuple<int, int>? charNGramRange, bool wordBoundaries, int minDf, double maxDf, int? maxFeatures, bool lowercase, CharNGramNorm norm, HashSet<string> stopWords, ITokenizer advancedTokenizer) |
Creates a new instance of CharNGramVectorizer. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Transform(IEnumerable<string> documents) |
Transforms documents to character n-gram vectors. |
void Fit(IEnumerable<string> documents) |
Fits the vectorizer to the corpus. |