IPromptableSegmentation<T>

Interfaces · AiDotNet.Interfaces

Interface for interactive, promptable segmentation models like SAM that accept user prompts (points, boxes, masks, text) to segment specific objects.

Implements: ISegmentationModel<T>

Known implementations: PromptableSegmentationBase<T>

For Beginners

Promptable segmentation is like pointing at something in a photo and having the model outline it for you.

Prompt types:

  • Points: Click on an object and the model segments it
  • Boxes: Draw a rectangle around an object for more precise segmentation
  • Masks: Provide a rough mask and the model refines it
  • Text: Describe what to segment (for models that support it)

Models implementing this interface:

  • SAM / SAM 2 (Meta, foundation model for segmentation)
  • SAM-HQ (high-quality boundaries)
  • SegGPT (in-context learning)
  • SEEM (multi-modal prompts including audio)

How It Works

Promptable segmentation models first encode an image, then accept various types of prompts to segment specific regions. This two-stage design allows efficient interactive use where the image is encoded once and multiple prompts can be processed quickly.

Properties

Property Summary
bool SupportsBoxPrompts { get; } Gets whether this model supports box prompts.
bool SupportsMaskPrompts { get; } Gets whether this model supports mask prompts.
bool SupportsPointPrompts { get; } Gets whether this model supports point prompts.
bool SupportsTextPrompts { get; } Gets whether this model supports text prompts.

Methods

Method Summary
List<PromptedSegmentationResult<T>> SegmentEverything() Generates masks for the entire image without any prompts (automatic mode).
PromptedSegmentationResult<T> SegmentFromBox(Tensor<T> box) Segments a region indicated by a bounding box prompt.
PromptedSegmentationResult<T> SegmentFromMask(Tensor<T> mask) Segments a region using a rough mask as a prompt.
PromptedSegmentationResult<T> SegmentFromPoints(Tensor<T> points, Tensor<T> labels) Segments a region indicated by point prompts.
void SetImage(Tensor<T> image) Encodes an image for subsequent prompted segmentation.