SigLIP<T>

Models & Types · AiDotNet.VisionLanguage.Encoders

SigLIP (Sigmoid Loss for Language-Image Pre-training) model for zero-shot classification and cross-modal retrieval with improved batch scaling.

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SigLIP<T>

Implements: IContrastiveVisionLanguageModel<T>

For Beginners

SigLIP is an improved version of CLIP. The main difference is in how it learns: instead of comparing every image with every text in a batch simultaneously (which requires lots of memory), SigLIP looks at each image-text pair one at a time and asks “do these match?”. This simple change makes training faster and results better.

Usage:

How It Works

SigLIP (Zhai et al., ICCV 2023) replaces CLIP’s softmax-based InfoNCE loss with a sigmoid loss that operates on individual image-text pairs independently. This removes the need for global batch normalization, enabling efficient training with very large batch sizes (up to 1M pairs). The model achieves 84.5% zero-shot on ImageNet with ViT-L/16@384.

Architecture:

  • Vision encoder: ViT with patch sizes 16 or 14, supporting multiple image resolutions
  • Text encoder: Transformer with causal attention, shared vocabulary
  • Sigmoid contrastive loss: Per-pair binary classification: positive = 1, negative = -1
  • Learnable temperature + bias: sigmoid(z * (sim/t + b)) where t and b are learned

Key Innovation: The sigmoid loss removes the softmax normalization across the full batch, which means each image-text pair is treated independently. This allows efficient distributed training without needing to gather all embeddings globally, and empirically gives better performance at scale.

References:

  • Paper: “Sigmoid Loss for Language Image Pre-Training” (Zhai et al., ICCV 2023)
  • Paper: “SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding” (2025)
  • Repository: https://github.com/google-research/big_vision (SigLIP components)

Example

// Create a SigLIP model for sigmoid-based contrastive learning
// with improved batch scaling and per-pair binary classification
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
            
// ONNX inference mode with pre-trained model
var model = new SigLIP<double>(architecture, "siglip.onnx");
            
// Training mode with native layers
var trainModel = new SigLIP<double>(architecture, new SigLIPOptions());

Constructors

Constructor Summary
SigLIP(NeuralNetworkArchitecture<T> architecture, SigLIPOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) Creates a SigLIP model in native training mode.
SigLIP(NeuralNetworkArchitecture<T> architecture, string imageEncoderModelPath, SigLIPOptions options) Creates a SigLIP model in native training mode.

Properties

Property Summary
T Temperature { get; }
int EmbeddingDimension { get; }
int MaxSequenceLength { get; }
int ProjectionDimension { get; }
int TextEmbeddingDimension { get; }

Methods

Method Summary
Dictionary<string, T> ZeroShotClassify(Tensor<T> image, string[] labels)
ModelMetadata<T> GetModelMetadata()
T ComputeSimilarity(Tensor<T> image, string text)
Tensor<T> EncodeImage(Tensor<T> image)
Tensor<T> EncodeText(string text)
Tensor<T>[] EncodeTexts(string[] texts)