TransformerEmbeddingNetwork<T>
Models & Types · AiDotNet.NeuralNetworks
A customizable Transformer-based embedding network. This serves as the high-performance foundation for modern sentence and document encoders.
Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → TransformerEmbeddingNetwork<T>
Implements: IEmbeddingModel<T>
Known derived types: BGE<T>, ColBERT<T>, InstructorEmbedding<T>, MatryoshkaEmbedding<T>, SGPT<T>, SPLADE<T>, SimCSE<T>
For Beginners
This is a “universal reading brain.” Transformers are the most powerful type of AI for understanding language because they can look at every word in a sentence at the same time and see how they all relate. This customizable version lets you decide how many layers of thinking the brain should have, and how it should summarize its thoughts into a final list of numbers (the embedding).
How It Works
This network provides a flexible implementation of the Transformer encoder architecture, enabling the generation of high-quality semantic embeddings. It supports multiple pooling strategies (Mean, Max, ClsToken) to aggregate token-level information.
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 128 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new TransformerEmbeddingNetwork<float>(architecture))
.Build(trainX, trainY);
var embedding = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
TransformerEmbeddingNetwork() |
Initializes a new instance with default architecture settings. |
TransformerEmbeddingNetwork(NeuralNetworkArchitecture<T> architecture, TransformerEmbeddingOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the TransformerEmbeddingNetwork. |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
|
int MaxTokens { get; } |
Methods
| Method | Summary |
|---|---|
Matrix<T> EmbedBatch(IEnumerable<string> texts) |
Encodes a collection of strings into a matrix of embeddings. |
ModelMetadata<T> GetModelMetadata() |
Returns metadata about the transformer network configuration. |
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts) |
|
Task<Vector<T>> EmbedAsync(string text) |
|
Vector<T> Embed(string text) |
Encodes a single string into a normalized summary vector. |