Word2Vec<T>
Models & Types · AiDotNet.NeuralNetworks
Word2Vec neural network implementation supporting both Skip-Gram and CBOW architectures.
Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → Word2Vec<T>
Implements: IEmbeddingModel<T>
For Beginners
Imagine you are learning a new language by looking at thousands of newspapers. You notice that the word “bark” often appears near “dog,” “tree,” and “loud.” You also notice that “meow” appears near “cat,” “kitten,” and “soft.” Word2Vec is an AI that does exactly this—it builds a “map” of words where words with similar meanings or contexts are placed close together. In this map, “dog” and “cat” might be neighbors, while “dog” and “spaceship” are on opposite ends.
How It Works
Word2Vec is a foundational technique in Natural Language Processing (NLP) that learns to map words to dense vectors of real numbers. These “embeddings” capture semantic and syntactic relationships based on the contexts in which words appear.
This implementation supports two main styles:
- Skip-Gram: Tries to guess the surrounding “context” words when given a single target word.
- CBOW: Tries to guess a single “target” word when given a group of surrounding context words.
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 50 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new Word2Vec<float>(architecture))
.Build(trainX, trainY);
var embedding = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
Word2Vec() |
Initializes a new instance with default architecture settings. |
Word2Vec(NeuralNetworkArchitecture<T> architecture, Word2VecOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the Word2Vec model. |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
|
int MaxTokens { get; } |
Methods
| Method | Summary |
|---|---|
Matrix<T> EmbedBatch(IEnumerable<string> texts) |
Encodes a collection of strings into a matrix of embedding vectors. |
ModelMetadata<T> GetModelMetadata() |
Retrieves detailed metadata about the Word2Vec model. |
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts) |
|
Task<Vector<T>> EmbedAsync(string text) |
|
Tensor<T> Forward(Tensor<T> input) |
Performs a forward pass through the network, typically to retrieve an embedding for given token IDs. |
Vector<T> Embed(string text) |
Encodes a single string into a normalized embedding vector by averaging its word vectors. |