OMGSeg<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Foundation
OMG-Seg: Is One Model Good Enough For All Segmentation?
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PanopticSegmentationBase<T> → OMGSeg<T>
For Beginners
OMG-Seg answers “yes” — one model handles over 10 segmentation tasks with only 70M trainable parameters. Instead of training separate models for semantic, instance, panoptic, video, interactive, and open-vocabulary segmentation, OMG-Seg uses task-specific queries to switch between tasks at inference time.
Common use cases:
- Multi-task segmentation systems needing many task types
- Resource-constrained environments where one model must serve all needs
- Research comparing different segmentation paradigms
- Production systems with diverse segmentation requirements
How It Works
Technical Details:
- Shared transformer backbone with task-specific query embeddings
- Each task type has its own set of queries that specialize during training
- Supports image (semantic, instance, panoptic) and video segmentation
- Open-vocabulary capability through text-conditioned queries
- Only 70M trainable parameters with frozen backbone
Reference: Li et al., “OMG-Seg: Is One Model Good Enough For All Segmentation?”, CVPR 2024.
Example
// Create an OMG-Seg model for multi-task segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new OMGSeg<double>(architecture, options: new OMGSegOptions { NumClasses = 150 });
// Or load a pre-trained ONNX model for unified segmentation
var onnxModel = new OMGSeg<double>(architecture, "omgseg.onnx", options: new OMGSegOptions { NumClasses = 150 });
Constructors
| Constructor | Summary |
|---|---|
OMGSeg(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, OMGSegOptions options) |
Initializes OMG-Seg in native (trainable) mode. |
OMGSeg(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, OMGSegOptions options) |
Initializes OMG-Seg in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this OMG-Seg model’s configuration. |
PanopticSegmentationResult<T> SegmentPanoptic(Tensor<T> image) |