CharNGramVectorizer<T>

Models & Types · AiDotNet.Preprocessing.TextVectorizers

Converts text documents to character-level n-gram feature vectors.

Inheritance: TextVectorizerBase<T> → CharNGramVectorizer<T>

For Beginners

Character n-grams capture patterns at the letter level:

  • “hello” with (2,3) n-grams produces: “he”, “el”, “ll”, “lo”, “hel”, “ell”, “llo”
  • Robust to spelling mistakes (“color” and “colour” share many character n-grams)
  • Works well for author identification and language detection
  • Can capture morphological patterns (prefixes, suffixes)

How It Works

CharNGramVectorizer creates features from character sequences rather than words. This is particularly useful for:

  • Handling misspellings and typos
  • Languages without clear word boundaries
  • Capturing subword patterns
  • Short text classification (tweets, SMS)

Constructors

Constructor Summary
CharNGramVectorizer(ValueTuple<int, int>? charNGramRange, bool wordBoundaries, int minDf, double maxDf, int? maxFeatures, bool lowercase, CharNGramNorm norm, HashSet<string> stopWords, ITokenizer advancedTokenizer) Creates a new instance of CharNGramVectorizer.

Methods

Method Summary
Matrix<T> Transform(IEnumerable<string> documents) Transforms documents to character n-gram vectors.
void Fit(IEnumerable<string> documents) Fits the vectorizer to the corpus.