SigLIPSO<T>

Models & Types · AiDotNet.VisionLanguage.Encoders

SigLIP-SO (Shape-Optimized SigLIP) 400M vision encoder widely used as a VLM backbone.

Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SigLIPSO<T>

Implements: IVisualEncoder<T>

For Beginners

SigLIP-SO is a shape-optimized version of SigLIP that carefully tunes the width and depth of the Vision Transformer for the 400M parameter budget. The resulting SO-400M/14 model has become one of the most popular vision backbones, used by LLaVA, PaliGemma, and many other multimodal models as their vision encoder. Default values follow the original paper settings.

How It Works

SigLIP-SO (Zhai et al., 2023) optimizes the ViT width/depth ratio for the 400M parameter budget. The resulting SO-400M/14 model produces high-quality visual features adopted by LLaVA, PaliGemma, and other popular VLMs. It uses sigmoid contrastive loss for training.

References:

  • Paper: “Sigmoid Loss for Language Image Pre-Training” (Zhai et al., ICCV 2023)

Example

// Create a SigLIP-SO model for shape-optimized visual features
// widely adopted as a VLM backbone by LLaVA, PaliGemma, etc.
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.TwoDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
            
// ONNX inference mode with pre-trained model
var model = new SigLIPSO<double>(architecture, "siglipso.onnx");
            
// Training mode with native layers
var trainModel = new SigLIPSO<double>(architecture, new SigLIPSOOptions());

Constructors

Constructor Summary
SigLIPSO(NeuralNetworkArchitecture<T> architecture, SigLIPSOOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer)
SigLIPSO(NeuralNetworkArchitecture<T> architecture, string modelPath, SigLIPSOOptions options)

Properties

Property Summary
int EmbeddingDimension { get; }

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> EncodeImage(Tensor<T> image)