SigLIP2Options

Options & Configuration · AiDotNet.VisionLanguage.Encoders

Configuration options for SigLIP 2 (Multilingual Vision-Language Encoders with Improved Semantic Understanding).

Inheritance: ModelOptions → ContrastiveEncoderOptions → SigLIP2Options

For Beginners

SigLIP 2 is an improved version of SigLIP that learns from three tasks simultaneously: (1) matching images with text, (2) generating image descriptions, and (3) predicting hidden parts of images. This multi-task learning produces better representations and supports many languages.

How It Works

SigLIP 2 (Tschannen et al., 2025) improves upon SigLIP by combining multiple training objectives: sigmoid contrastive loss, autoregressive captioning loss, and self-supervised masked image modeling. This multi-objective approach produces encoders with better semantic understanding while maintaining efficient scaling. The model also supports 32+ languages through a multilingual text encoder.

Key Differences from SigLIP:

  • Multi-objective training: Contrastive + captioning + self-supervised losses
  • Captioning decoder: Lightweight autoregressive decoder for generating image descriptions
  • Masked image modeling: Self-supervised patch prediction for better spatial understanding
  • Multilingual: Extended vocabulary with 32+ language support via mPaLM tokenizer
  • Online data curation: Dynamic mixing of data sources during training

Constructors

Constructor Summary
SigLIP2Options() Initializes default SigLIP 2 options.
SigLIP2Options(SigLIP2Options other) Initializes a new instance by copying from another instance.

Properties

Property Summary
ContrastiveLossType LossType { get; set; } Gets or sets the contrastive loss type (default: Sigmoid for SigLIP family).
bool IncludeCaptioningDecoder { get; set; } Gets or sets whether to include the captioning decoder in inference mode.
bool Multilingual { get; set; } Gets or sets whether multilingual text encoding is enabled.
double CaptioningLossWeight { get; set; } Gets or sets the weight for the captioning loss in multi-objective training.
double MimMaskRatio { get; set; } Gets or sets the mask ratio for masked image modeling (fraction of patches masked).
double SelfSupervisedLossWeight { get; set; } Gets or sets the weight for the self-supervised masked image modeling loss.
double SigmoidBias { get; set; } Gets or sets the bias term for sigmoid contrastive loss.
int CaptioningDecoderDim { get; set; } Gets or sets the hidden dimension for the captioning decoder.
int MaxCaptionLength { get; set; } Gets or sets the maximum caption length for the captioning decoder.
int MimDecoderDim { get; set; } Gets or sets the MIM decoder dimension for predicting masked patch features.
int NumCaptioningDecoderHeads { get; set; } Gets or sets the number of attention heads in the captioning decoder.
int NumCaptioningDecoderLayers { get; set; } Gets or sets the number of captioning decoder layers.
int NumMimDecoderLayers { get; set; } Gets or sets the number of MIM decoder layers.