IVisualGroundingModel<T>

Interfaces · AiDotNet.VisionLanguage.Interfaces

Interface for visual grounding models that localize objects or regions from natural language descriptions.

Implements: IVisualEncoder<T>

Known implementations: DINOX<T>, FerretV2<T>, Ferret<T>, Groma<T>, GroundedSAM2<T>, GroundingDINO15<T>, GroundingDINO<T>, OWLViT<T>, OWLv2<T>, Shikra<T>

How It Works

Visual grounding models take an image and a text query and produce bounding boxes, segmentation masks, or region proposals for the objects/regions described by the text. Architectures include:

  • Grounding DINO: DINO detector + grounded pre-training for open-set detection
  • Ferret: Spatial-aware visual sampler for free-form region inputs
  • OWL-ViT/OWLv2: Open-vocabulary detection via CLIP-aligned ViT

Properties

Property Summary
int MaxDetections { get; } Gets the maximum number of detections the model can produce per image.

Methods

Method Summary
Tensor<T> DetectObjects(Tensor<T> image, IReadOnlyList<string> categories) Detects objects in the image matching a set of category descriptions.
Tensor<T> GroundText(Tensor<T> image, string textQuery) Grounds a text query in the image, producing bounding box coordinates.