SigLIP<T>
Models & Types · AiDotNet.VisionLanguage.Encoders
SigLIP (Sigmoid Loss for Language-Image Pre-training) model for zero-shot classification and cross-modal retrieval with improved batch scaling.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SigLIP<T>
Implements: IContrastiveVisionLanguageModel<T>
For Beginners
SigLIP is an improved version of CLIP. The main difference is in how it learns: instead of comparing every image with every text in a batch simultaneously (which requires lots of memory), SigLIP looks at each image-text pair one at a time and asks “do these match?”. This simple change makes training faster and results better.
Usage:
How It Works
SigLIP (Zhai et al., ICCV 2023) replaces CLIP’s softmax-based InfoNCE loss with a sigmoid loss that operates on individual image-text pairs independently. This removes the need for global batch normalization, enabling efficient training with very large batch sizes (up to 1M pairs). The model achieves 84.5% zero-shot on ImageNet with ViT-L/16@384.
Architecture:
- Vision encoder: ViT with patch sizes 16 or 14, supporting multiple image resolutions
- Text encoder: Transformer with causal attention, shared vocabulary
- Sigmoid contrastive loss: Per-pair binary classification: positive = 1, negative = -1
- Learnable temperature + bias: sigmoid(z * (sim/t + b)) where t and b are learned
Key Innovation: The sigmoid loss removes the softmax normalization across the full batch, which means each image-text pair is treated independently. This allows efficient distributed training without needing to gather all embeddings globally, and empirically gives better performance at scale.
References:
- Paper: “Sigmoid Loss for Language Image Pre-Training” (Zhai et al., ICCV 2023)
- Paper: “SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding” (2025)
- Repository:
https://github.com/google-research/big_vision(SigLIP components)
Example
// Create a SigLIP model for sigmoid-based contrastive learning
// with improved batch scaling and per-pair binary classification
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new SigLIP<double>(architecture, "siglip.onnx");
// Training mode with native layers
var trainModel = new SigLIP<double>(architecture, new SigLIPOptions());
Constructors
| Constructor | Summary |
|---|---|
SigLIP(NeuralNetworkArchitecture<T> architecture, SigLIPOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
Creates a SigLIP model in native training mode. |
SigLIP(NeuralNetworkArchitecture<T> architecture, string imageEncoderModelPath, SigLIPOptions options) |
Creates a SigLIP model in native training mode. |
Properties
| Property | Summary |
|---|---|
T Temperature { get; } |
|
int EmbeddingDimension { get; } |
|
int MaxSequenceLength { get; } |
|
int ProjectionDimension { get; } |
|
int TextEmbeddingDimension { get; } |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> ZeroShotClassify(Tensor<T> image, string[] labels) |
|
ModelMetadata<T> GetModelMetadata() |
|
T ComputeSimilarity(Tensor<T> image, string text) |
|
Tensor<T> EncodeImage(Tensor<T> image) |
|
Tensor<T> EncodeText(string text) |
|
Tensor<T>[] EncodeTexts(string[] texts) |