SimCSE<T>
Models & Types · AiDotNet.NeuralNetworks
SimCSE (Simple Contrastive Learning of Sentence Embeddings) neural network implementation.
Inheritance: NeuralNetworkBase<T> → DeclaredModelLayoutBase<T> → TextEmbeddingModelLayoutBase<T> → TransformerEmbeddingNetwork<T> → SimCSE<T>
For Beginners
Imagine you’re trying to recognize a friend in a crowded room. Even if they are wearing a hat, glasses, or a scarf (like “dropout” noise), they are still the same person. SimCSE trains the model by showing it the same sentence twice with different “masks” and telling it: “this is the same sentence.” This helps the model learn the true, deep meaning of the sentence that stays constant regardless of small changes.
How It Works
SimCSE is a state-of-the-art framework for learning sentence embeddings. It uses a contrastive learning objective to pull semantically similar sentences together and push dissimilar ones apart. Its most famous variant is unsupervised, using different dropout masks on the same sentence as a minimal data augmentation.
Example
var architecture = new NeuralNetworkArchitecture<float>(inputFeatures: 8, outputSize: 4);
var input = Tensor<float>.CreateRandom(new[] { 1, 128 });
var trainX = Tensor<float>.CreateRandom(4, 8);
var trainY = Tensor<float>.CreateRandom(4, 2);
var result = new AiModelBuilder<float, Tensor<float>, Tensor<float>>()
.ConfigureModel(new SimCSE<float>(architecture))
.Build(trainX, trainY);
var embedding = result.Predict(input);
Constructors
| Constructor | Summary |
|---|---|
SimCSE() |
Initializes a new instance with default architecture settings. |
SimCSE(NeuralNetworkArchitecture<T> architecture, SimCSEOptions options, ITokenizer tokenizer, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction) |
Initializes a new instance of the SimCSE model. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Retrieves detailed metadata about the SimCSE configuration. |
Task<Matrix<T>> EmbedBatchAsync(IEnumerable<string> texts) |
|
Task<Vector<T>> EmbedAsync(string text) |
|
Vector<T> Embed(string text) |