PretrainedEmbeddingsVectorizer<T>
Models & Types ยท AiDotNet.Preprocessing.TextVectorizers
Converts text documents to vectors using pre-trained word embeddings loaded from files.
Inheritance: TextVectorizerBase<T> โ PretrainedEmbeddingsVectorizer<T>
For Beginners
Use this when you have pre-trained embeddings:
- Download pre-trained vectors (GloVe, Word2Vec, FastText) from the web
- Load them once and use for any text classification or similarity task
- Much faster than training your own embeddings
- Works great for most NLP tasks
How It Works
This vectorizer loads pre-trained word embeddings from standard file formats (Word2Vec text, GloVe, FastText) and uses them to create document representations.
Supported file formats:
- Word2Vec text format: First line contains vocab_size and vector_size, followed by word vectors
- GloVe format: Each line contains word followed by vector values (no header)
- FastText format: Same as Word2Vec text format
Constructors
| Constructor | Summary |
|---|---|
PretrainedEmbeddingsVectorizer(Dictionary<string, double[]> wordVectors, Word2VecAggregation aggregation, double oovVector, bool lowercase, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) |
Creates a new instance from already-loaded word vectors. |
PretrainedEmbeddingsVectorizer(string embeddingsPath, PretrainedFormat format, int? maxVocabSize, Word2VecAggregation aggregation, double oovVector, bool lowercase, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) |
Creates a new instance of PretrainedEmbeddingsVectorizer by loading embeddings from a file. |
Properties
| Property | Summary |
|---|---|
Dictionary<string, double[]> WordVectors { get; } |
Gets the loaded word vectors. |
bool IsFitted { get; } |
|
int FeatureCount { get; } |
|
int VectorSize { get; } |
Gets the vector dimensionality. |
Methods
| Method | Summary |
|---|---|
Matrix<T> Transform(IEnumerable<string> documents) |
Transforms documents to dense vectors using pre-trained embeddings. |
double[] GetWordVector(string word) |
Gets the vector for a specific word. |
void Fit(IEnumerable<string> documents) |
Fitting is not required for pre-trained embeddings vectorizer. |