MaskDINO<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Foundation
Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PanopticSegmentationBase<T> → MaskDINO<T>
For Beginners
Mask DINO extends the powerful DINO object detector with a mask prediction branch, creating a unified architecture that handles object detection, instance segmentation, panoptic segmentation, and semantic segmentation all in one model. Instead of building separate models for each task, Mask DINO uses a shared backbone and query-based transformer to do everything.
Common use cases:
- Joint object detection + instance segmentation
- Panoptic segmentation (detecting all things and stuff)
- Research requiring a unified detection-segmentation framework
- Production systems needing both detection boxes and segmentation masks
How It Works
Technical Details:
- Built on DINO detector with deformable attention transformer encoder-decoder
- Adds a mask branch using dot product between query embeddings and pixel embeddings
- Unified query matching for both box and mask predictions via Hungarian matching
- Backbone: ResNet-50 or Swin-L
- Achieves 54.5 AP on COCO instance, 59.4 PQ on COCO panoptic
Reference: Li et al., “Mask DINO: Towards A Unified Transformer-based Framework for Object Detection and Segmentation”, CVPR 2023.
Example
// Create a MaskDINO model for joint detection and segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 80);
var model = new MaskDINO<double>(architecture, options: new MaskDINOOptions { NumClasses = 80 });
// Or load a pre-trained ONNX model for unified detection-segmentation
var onnxModel = new MaskDINO<double>(architecture, "maskdino.onnx", options: new MaskDINOOptions { NumClasses = 80 });
Constructors
| Constructor | Summary |
|---|---|
MaskDINO(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, MaskDINOOptions options) |
Initializes Mask DINO in native (trainable) mode. |
MaskDINO(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, MaskDINOOptions options) |
Initializes Mask DINO in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this Mask DINO model’s configuration. |
PanopticSegmentationResult<T> SegmentPanoptic(Tensor<T> image) |