SPLADE<T>

Models & Types · AiDotNet.NeuralNetworks

SPLADE (Sparse Lexical and Expansion Model) neural network implementation. Maps text to a high-dimensional sparse vector in the vocabulary space.

Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → TransformerEmbeddingNetwork<T> → SPLADE<T>

For Beginners

Imagine a dictionary with 30,000 words. For every sentence, SPLADE creates a giant list of 30,000 numbers, but almost all of them are zero. It only puts numbers next to the words that are actually important to the meaning.

How It Works

SPLADE is a sparse retrieval model that learns to represent documents and queries as sparse vectors over the vocabulary. It uses a log-saturation effect and sparsity regularization (e.g., FLOPs or L1) to learn lexical expansion and term importance.

The “Expansion” part is the most interesting: if you say “The smartphone is fast,” SPLADE might automatically put a number next to the word “Apple” or “Android” in its dictionary, even if you didn’t say them. This helps it find relevant documents that use different words.

Example

var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
    .ConfigureModel(new SPLADE<float>(architecture))
    .Build(trainX, trainY);
var sparse = result.Predict(input);

Constructors

Constructor Summary
SPLADE() Initializes a new instance of the SPLADE model.
SPLADE(NeuralNetworkArchitecture<T> architecture, SPLADEOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction)

Methods

Method Summary
ModelMetadata<T> GetModelMetadata() Retrieves detailed metadata about the SPLADE configuration.
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts)
Task<Vector<T>> EmbedAsync(string text)
Vector<T> Embed(string text) Encodes text into a high-dimensional sparse lexical representation.