SPLADE<T>
Models & Types · AiDotNet.NeuralNetworks
SPLADE (Sparse Lexical and Expansion Model) neural network implementation. Maps text to a high-dimensional sparse vector in the vocabulary space.
Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → TransformerEmbeddingNetwork<T> → SPLADE<T>
For Beginners
Imagine a dictionary with 30,000 words. For every sentence, SPLADE creates a giant list of 30,000 numbers, but almost all of them are zero. It only puts numbers next to the words that are actually important to the meaning.
How It Works
SPLADE is a sparse retrieval model that learns to represent documents and queries as sparse vectors over the vocabulary. It uses a log-saturation effect and sparsity regularization (e.g., FLOPs or L1) to learn lexical expansion and term importance.
The “Expansion” part is the most interesting: if you say “The smartphone is fast,” SPLADE might automatically put a number next to the word “Apple” or “Android” in its dictionary, even if you didn’t say them. This helps it find relevant documents that use different words.
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 256 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SPLADE<float>(architecture))
.Build(trainX, trainY);
var sparse = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
SPLADE() |
Initializes a new instance of the SPLADE model. |
SPLADE(NeuralNetworkArchitecture<T> architecture, SPLADEOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Retrieves detailed metadata about the SPLADE configuration. |
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts) |
|
Task<Vector<T>> EmbedAsync(string text) |
|
Vector<T> Embed(string text) |
Encodes text into a high-dimensional sparse lexical representation. |