SegmentationBuilderExtensions

Helpers & Utilities · AiDotNet

Extension methods for image segmentation operations through the AiModelBuilder facade.

For Beginners

These methods let you perform segmentation on images after configuring a model with ConfigureModel. Each method detects what kind of segmentation model you configured and calls the appropriate interface.

Usage pattern:

Methods

Method Summary
InstanceSegmentationResult<T> SegmentInstances(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Detects and segments individual object instances in an image.
List<PromptedSegmentationResult<T>> SegmentEverything(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder) Automatically segments everything in the image without prompts.
List<ReferringSegmentationResult<T>> SegmentVideoFromExpression(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> frames, string expression) Segments and tracks objects in a video from a natural language description.
MedicalSegmentationResult<T> SegmentMedicalFewShot(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> queryImage, Tensor<T> supportImages, Tensor<T> supportMasks) Segments using few-shot examples (for models like UniverSeg, MedSAM).
MedicalSegmentationResult<T> SegmentMedicalSlice(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> slice) Segments a 2D medical image slice (CT, MRI, X-ray, etc.).
MedicalSegmentationResult<T> SegmentMedicalVolume(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> volume) Segments a 3D medical volume (full CT or MRI scan).
OpenVocabSegmentationResult<T> SegmentWithTextClasses(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image, IReadOnlyList<string> classNames) Segments objects described by text class names (open-vocabulary).
OpenVocabSegmentationResult<T> SegmentWithTextPrompt(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image, string prompt) Segments objects matching a single text prompt (grounded segmentation).
PanopticSegmentationResult<T> SegmentPanoptic(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Performs panoptic segmentation (unified semantic + instance) on an image.
PromptedSegmentationResult<T> SegmentFromBox(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> box) Segments the object inside a bounding box.
PromptedSegmentationResult<T> SegmentFromMask(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> mask) Refines a rough mask into a precise segmentation.
PromptedSegmentationResult<T> SegmentFromPoints(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> points, Tensor<T> labels) Segments the region indicated by point prompts (click to segment).
ReferringSegmentationResult<T> SegmentFromConversation(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image, IReadOnlyList<ValueTuple<string, string>> conversationHistory, string currentQuery) Segments objects from a multi-turn conversational context.
ReferringSegmentationResult<T> SegmentFromExpression(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image, string expression) Segments objects described by a natural language expression with reasoning.
Tensor<T> GetSemanticClassMap(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Gets the per-pixel class map (argmax of logits) from a semantic segmentation model.
Tensor<T> GetSemanticProbabilities(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Gets per-pixel class probability maps from a semantic segmentation model.
Tensor<T> SegmentSemantic(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Produces raw per-pixel class logits from a semantic segmentation model.
ValueTuple<int, int> GetSegmentationInputSize(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder) Gets the expected input dimensions of the configured segmentation model.
VideoSegmentationResult<T> PropagateSegmentation(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> frame) Propagates tracked object masks to the next video frame.
bool IsSegmentationOnnxMode(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder) Gets whether the configured segmentation model is running in ONNX mode.
int GetSegmentationClassCount(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder) Gets the number of segmentation classes the configured model predicts.
void CorrectSegmentationMask(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, int objectId, Tensor<T> correctionMask) Corrects a tracked object’s mask at the current frame.
void InitializeVideoTracking(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> firstFrame, Tensor<T> masks, int[] objectIds) Initializes video object tracking with first-frame masks.
void ResetSegmentationTracking(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder) Resets the video tracking state and memory.
void SetSegmentationImage(AiModelBuilder<T, Tensor<T>, Tensor<T>> builder, Tensor<T> image) Encodes an image for subsequent prompt-based segmentation.