IContrastiveVisionLanguageModel<T>
Interfaces · AiDotNet.VisionLanguage.Interfaces
Interface for contrastive vision-language models that align image and text embeddings in a shared space for zero-shot classification and cross-modal retrieval.
Implements: ITextEncoder<T>, IVisualEncoder<T>
Known implementations: ALIGN<T>, BASIC<T>, BiomedCLIP<T>, CLIPA<T>, DFNCLIP<T>, DeCLIP<T>, EVACLIP<T>, FLIP<T>, LLM2CLIP<T>, LiT<T>, MedCLIP<T>, MetaCLIP<T>, OpenCLIP<T>, RegionCLIP<T>, RemoteCLIP<T>, SigLIP2<T>, SigLIP<T>
For Beginners
Think of this like a universal translator between images and text. The model learns that a photo of a dog and the text “a photo of a dog” should have similar representations. This means you can search for images using text, or classify images by comparing them with text descriptions - without training on specific categories.
How It Works
Contrastive VLMs (like CLIP, SigLIP, ALIGN) learn to place matching image-text pairs close together in a shared embedding space while pushing non-matching pairs apart. This enables:
- Zero-shot classification: Classify images using text descriptions without training
- Image-text retrieval: Find images matching a text query, or texts describing an image
- Similarity scoring: Measure how well an image matches a text description
Properties
| Property | Summary |
|---|---|
T Temperature { get; } |
Gets the temperature parameter used for similarity scaling. |
int ProjectionDimension { get; } |
Gets the dimensionality of the shared projection space. |
Methods
| Method | Summary |
|---|---|
Dictionary<string, T> ZeroShotClassify(Tensor<T> image, string[] labels) |
Performs zero-shot image classification by comparing the image with text labels. |
T ComputeSimilarity(Tensor<T> image, string text) |
Computes the similarity score between an image and a text description. |