IOpenVocabSegmentation<T>

Interfaces · AiDotNet.Interfaces

Interface for open-vocabulary segmentation models that segment objects from text descriptions without being limited to a fixed set of classes.

Implements: ISegmentationModel<T>

Known implementations: OpenVocabSegmentationBase<T>

For Beginners

Traditional segmentation models can only recognize objects they were trained on (e.g., “car”, “person”, “dog”). Open-vocabulary models can segment anything you describe in words — even things they’ve never seen during training.

Example: You can ask for “red coffee mug on the table” or “person wearing a blue hat” and the model will try to segment exactly that, even if it was never trained on those specific combinations.

Models implementing this interface:

  • SAN (CVPR 2023, side adapter on CLIP)
  • CAT-Seg (CVPR 2024, cost aggregation)
  • SED (CVPR 2024, simple encoder-decoder)
  • Open-Vocabulary SAM (ECCV 2024, SAM + CLIP)
  • Grounded SAM 2 (grounding DINO + SAM 2)
  • Mask-Adapter (CVPR 2025, mask-level adaptation)

How It Works

Open-vocabulary segmentation models use vision-language understanding (typically CLIP-based) to segment objects described by arbitrary text. Unlike traditional models trained on a fixed set of classes, these models can segment any concept expressible in natural language.

Properties

Property Summary
int MaxCategories { get; } Gets the maximum number of text categories that can be queried simultaneously.
int MaxPromptLength { get; } Gets the maximum text prompt length in tokens.

Methods

Method Summary
OpenVocabSegmentationResult<T> SegmentWithPrompt(Tensor<T> image, string prompt) Segments an image using a single text prompt for grounded segmentation.
OpenVocabSegmentationResult<T> SegmentWithText(Tensor<T> image, IReadOnlyList<string> classNames) Segments an image using text descriptions of target classes.