PretrainedEmbeddingsVectorizer<T>

Models & Types ยท AiDotNet.Preprocessing.TextVectorizers

Converts text documents to vectors using pre-trained word embeddings loaded from files.

Inheritance: TextVectorizerBase<T> โ†’ PretrainedEmbeddingsVectorizer<T>

For Beginners

Use this when you have pre-trained embeddings:

  • Download pre-trained vectors (GloVe, Word2Vec, FastText) from the web
  • Load them once and use for any text classification or similarity task
  • Much faster than training your own embeddings
  • Works great for most NLP tasks

How It Works

This vectorizer loads pre-trained word embeddings from standard file formats (Word2Vec text, GloVe, FastText) and uses them to create document representations.

Supported file formats:

  • Word2Vec text format: First line contains vocab_size and vector_size, followed by word vectors
  • GloVe format: Each line contains word followed by vector values (no header)
  • FastText format: Same as Word2Vec text format

Constructors

Constructor Summary
PretrainedEmbeddingsVectorizer(Dictionary<string, double[]> wordVectors, Word2VecAggregation aggregation, double oovVector, bool lowercase, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) Creates a new instance from already-loaded word vectors.
PretrainedEmbeddingsVectorizer(string embeddingsPath, PretrainedFormat format, int? maxVocabSize, Word2VecAggregation aggregation, double oovVector, bool lowercase, Func<string, IEnumerable<string>> tokenizer, HashSet<string> stopWords, ITokenizer advancedTokenizer) Creates a new instance of PretrainedEmbeddingsVectorizer by loading embeddings from a file.

Properties

Property Summary
Dictionary<string, double[]> WordVectors { get; } Gets the loaded word vectors.
bool IsFitted { get; }
int FeatureCount { get; }
int VectorSize { get; } Gets the vector dimensionality.

Methods

Method Summary
Matrix<T> Transform(IEnumerable<string> documents) Transforms documents to dense vectors using pre-trained embeddings.
double[] GetWordVector(string word) Gets the vector for a specific word.
void Fit(IEnumerable<string> documents) Fitting is not required for pre-trained embeddings vectorizer.