IVisualEncoder<T>

Interfaces · AiDotNet.VisionLanguage.Interfaces

Interface for vision encoders that extract feature representations from images.

Known implementations: DINOv2<T>, DINOv3<T>, Florence2<T>, IContrastiveVisionLanguageModel<T>, IGenerativeVisionLanguageModel<T>, IImageEditingVLM<T>, IVisionLanguageFusionModel<T>, IVisualGroundingModel<T>, InternViT<T>, PerceptionEncoder<T>, RADIOv25<T>, SAM<T>, SigLIPSO<T>, ViT<T>

For Beginners

A visual encoder is like a “feature extractor” for images. It reads an image and produces a list of numbers (an embedding) that summarizes what the image contains. Two similar images will have similar embeddings, making it easy for other models to compare and reason about images.

How It Works

Visual encoders transform raw image data (pixel tensors) into compact feature representations (embeddings) that capture semantic content. These embeddings can be used for:

  • Image classification (zero-shot or fine-tuned)
  • Image-text similarity search and retrieval
  • Visual question answering (as input to downstream VLMs)
  • Object detection and segmentation (as backbone features)

Properties

Property Summary
int EmbeddingDimension { get; } Gets the dimensionality of the output embedding space.
int ImageChannels { get; } Gets the number of image channels expected (typically 3 for RGB).
int ImageSize { get; } Gets the expected input image size (height and width in pixels).

Methods

Method Summary
Tensor<T> EncodeImage(Tensor<T> image) Extracts a visual embedding from an image tensor.