MatryoshkaEmbedding<T>

Models & Types · AiDotNet.NeuralNetworks

Matryoshka Representation Learning (MRL) neural network implementation. Learns nested embeddings where smaller prefixes of the full vector are valid representations.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → TransformerEmbeddingNetwork<T> → MatryoshkaEmbedding<T>

For Beginners

Imagine a Russian nesting doll (a Matryoshka). Inside the big doll is a smaller one, and inside that is an even smaller one. MRL works the same way: it creates a long list of numbers to describe a sentence, but it makes sure that the first few numbers are a “perfect miniature” of the whole meaning. This lets you use a tiny list for a fast search and the full list when you need total accuracy.

How It Works

Matryoshka Representation Learning (MRL) is a technique that enables a single model to adapt its embedding dimension to the requirements of the downstream task. It optimizes for multiple dimensions simultaneously, ensuring high accuracy even when using truncated vector prefixes.

Example

var architecture = new NeuralNetworkArchitecture<float>(
    inputType: InputType.OneDimensional,
    taskType: NeuralNetworkTaskType.Regression,
    inputSize: 1536,
    outputSize: 1536);
var input = Tensor<float>.CreateRandom(new[] { 1, 512 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new MatryoshkaEmbedding<float>(architecture))
    .Build(trainX, trainY);
var embedding = result.Predict(input);

Constructors

Constructor Summary
MatryoshkaEmbedding() Initializes a new instance with default architecture settings.
MatryoshkaEmbedding(NeuralNetworkArchitecture<T> architecture, MatryoshkaEmbeddingOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) Initializes a new instance of the MatryoshkaEmbedding model.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata() Retrieves metadata about the Matryoshka configuration.
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts)
Task<Vector<T>> EmbedAsync(string text)
Vector<T> Embed(string text)
Vector<T> EmbedResized(string text, int dimension) Encodes text into a truncated and re-normalized embedding of the requested dimension.