SEEM<T>

Models & Types · AiDotNet.ComputerVision.Segmentation.Interactive

SEEM: Segment Everything Everywhere All at Once.

Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PromptableSegmentationBase<T> → SEEM<T>

For Beginners

Multi-modal interactive segmentation. Text-guided, click-guided, and box-guided segmentation.

Common use cases:

  • Multi-modal interactive segmentation
  • Text-guided, click-guided, and box-guided segmentation
  • Referring expression segmentation
  • Open-vocabulary segmentation

How It Works

Technical Details:

  • Joint visual-semantic decoder for multi-modal prompts
  • Supports text, point, box, and scribble prompts simultaneously
  • Focal-Tiny or Focal-Large backbone
  • Unified architecture for interactive + automatic segmentation

Reference: Zou et al., “Segment Everything Everywhere All at Once”, NeurIPS 2023.

Example

// Create a SEEM model for multi-modal interactive segmentation
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.MultiClassClassification,
    inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new SEEM<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SEEM<double>(architecture,
    options: new SEEMOptions { NumClasses = 133, DropRate = 0.0, ModelSize = SEEMModelSize.Tiny });
            
// Or load a pre-trained ONNX model for text and click-guided segmentation
var onnxModel = new SEEM<double>(architecture, "seem.onnx");

Constructors

Constructor Summary
SEEM(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SEEMOptions options) Initializes SEEM in native (trainable) mode.
SEEM(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SEEMOptions options) Initializes SEEM in ONNX (inference-only) mode.

Properties

Property Summary
bool SupportsTextPrompts { get; }

Methods

Method Summary
List<PromptedSegmentationResult<T>> SegmentEverything()
ModelMetadata<T> GetModelMetadata() Collects metadata describing this model’s configuration.
PromptedSegmentationResult<T> SegmentFromBox(Tensor<T> box)
PromptedSegmentationResult<T> SegmentFromMask(Tensor<T> mask)
PromptedSegmentationResult<T> SegmentFromPoints(Tensor<T> points, Tensor<T> labels)
void SetImage(Tensor<T> image)