OpenVocabSegmentationBase<T>

Base Classes · AiDotNet.ComputerVision.Segmentation.Common

Abstract base class for open-vocabulary segmentation models that segment objects described by arbitrary text without being limited to a fixed class set.

Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → OpenVocabSegmentationBase<T>

Implements: IOpenVocabSegmentation<T>

Known derived types: CATSeg<T>, MaskAdapter<T>, OpenVocabSAM<T>, SAN<T>, SED<T>

For Beginners

Traditional segmentation models only recognize objects they were trained on (e.g., “car”, “person”). Open-vocabulary models use language understanding (typically CLIP) to segment anything you describe in text, even novel concepts.

Models extending this base class: SAN, CAT-Seg, SED, Open-Vocabulary SAM, Grounded SAM 2, Mask-Adapter.

Properties

Property Summary
ITextEncoder<T> TextEncoder { get; } Gets the text encoder whose embeddings the pixel embeddings are compared with.
int MaxCategories { get; }
int MaxPromptLength { get; }

Methods

Method Summary
OpenVocabSegmentationResult<T> SegmentWithPrompt(Tensor<T> image, string prompt)
OpenVocabSegmentationResult<T> SegmentWithText(Tensor<T> image, IReadOnlyList<string> classNames)
Tensor<T> ForwardForTraining(Tensor<T> input) The training forward: the base layer walk (seed wiring, checkpointing and weight streaming included) followed by ClassifyPixels, so training fits the same per-name logits that Predict returns rather than the pixel embeddings the layers en…
Tensor<T> ForwardGpu(Tensor<T> input) The GPU-resident forward: the base layer walk followed by ClassifyPixels, the same output as Forward and Predict instead of the decoder’s pixel embeddings.