FastTextVectorizer<T>
Models & Types · AiDotNet.Preprocessing.TextVectorizers
Converts text documents to dense vectors using FastText-style subword embeddings.
Inheritance: TextVectorizerBase<T> → FastTextVectorizer<T>
For Beginners
FastText is Word2Vec that understands word parts:
- Can generate vectors for words it has never seen before
- “unhappiness” can be understood from “un-”, “happy”, “-ness” patterns
- Better for languages with rich morphology (German, Turkish, Finnish)
- Handles typos and spelling variations better than Word2Vec
How It Works
FastText extends Word2Vec by representing each word as a bag of character n-grams. This allows the model to:
- Handle out-of-vocabulary (OOV) words by composing subword vectors
- Capture morphological information (prefixes, suffixes, roots)
- Be more robust to spelling variations and typos
For example, “where” with n-grams (3,6) includes: “<wh”, “whe”, “her”, “ere”, “re>”,
“<whe”, “wher”, “here”, “ere>”, “<wher”, “where”, “here>”, “<where”, “where>”, “
Constructors
| Constructor | Summary |
|---|---|
FastTextVectorizer(int vectorSize, int windowSize, int minCount, int epochs, double learningRate, int negativeSamples, ValueTuple<int, int>? subwordNGramRange, int buckets, Word2VecAggregation aggregation, bool lowercase, int? randomState, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) |
Creates a new instance of FastTextVectorizer. |
Properties
| Property | Summary |
|---|---|
Dictionary<string, double[]> WordVectors { get; } |
Gets the learned word vectors for known words. |
bool IsFitted { get; } |
|
int FeatureCount { get; } |
|
int VectorSize { get; } |
Gets the vector dimensionality. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Transform(IEnumerable<string> documents) |
Transforms documents to dense vectors by aggregating word vectors. |
double[] GetWordVector(string word) |
Gets the vector for a word, computing from subwords if unknown. |
void Fit(IEnumerable<string> documents) |
Trains FastText embeddings on the corpus. |