SigLIPSO<T>
Models & Types · AiDotNet.VisionLanguage.Encoders
SigLIP-SO (Shape-Optimized SigLIP) 400M vision encoder widely used as a VLM backbone.
Inheritance: NeuralNetworkBase<T> → VisionLanguageModelBase<T> → SigLIPSO<T>
Implements: IVisualEncoder<T>
For Beginners
SigLIP-SO is a shape-optimized version of SigLIP that carefully tunes the width and depth of the Vision Transformer for the 400M parameter budget. The resulting SO-400M/14 model has become one of the most popular vision backbones, used by LLaVA, PaliGemma, and many other multimodal models as their vision encoder. Default values follow the original paper settings.
How It Works
SigLIP-SO (Zhai et al., 2023) optimizes the ViT width/depth ratio for the 400M parameter budget. The resulting SO-400M/14 model produces high-quality visual features adopted by LLaVA, PaliGemma, and other popular VLMs. It uses sigmoid contrastive loss for training.
References:
- Paper: “Sigmoid Loss for Language Image Pre-Training” (Zhai et al., ICCV 2023)
Example
// Create a SigLIP-SO model for shape-optimized visual features
// widely adopted as a VLM backbone by LLaVA, PaliGemma, etc.
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.TwoDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 224, inputWidth: 224, inputDepth: 3, outputSize: 512);
// ONNX inference mode with pre-trained model
var model = new SigLIPSO<double>(architecture, "siglipso.onnx");
// Training mode with native layers
var trainModel = new SigLIPSO<double>(architecture, new SigLIPSOOptions());
Constructors
| Constructor | Summary |
|---|---|
SigLIPSO(NeuralNetworkArchitecture<T> architecture, SigLIPSOOptions options, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer) |
|
SigLIPSO(NeuralNetworkArchitecture<T> architecture, string modelPath, SigLIPSOOptions options) |
Properties
| Property | Summary |
|---|---|
int EmbeddingDimension { get; } |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> EncodeImage(Tensor<T> image) |