UNINEXT<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Foundation
UNINEXT: Universal Instance Perception as Object Discovery and Retrieval.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PanopticSegmentationBase<T> → UNINEXT<T>
For Beginners
UNINEXT reformulates over 10 different instance perception tasks into a unified “discover and retrieve” framework. Whether you need object detection, instance segmentation, single-object tracking, multi-object tracking, video instance segmentation, or referring expression segmentation, UNINEXT handles them all with one model by using task-specific prompt embeddings.
Common use cases:
- Multi-task instance perception (detection + segmentation + tracking)
- Video object segmentation with tracking
- Referring expression comprehension (find object from text description)
- SOTA results on 20+ benchmarks simultaneously
How It Works
Technical Details:
- Unified query representation for all instance perception tasks
- Task-specific prompt embeddings select which task to perform
- Backbone: ResNet-50, Swin-L, or ViT-H
- Deformable transformer encoder-decoder
- SOTA on 20+ benchmarks across detection, segmentation, tracking
Reference: Yan et al., “Universal Instance Perception as Object Discovery and Retrieval”, CVPR 2023.
Example
// Create a UNINEXT model for unified instance perception (detection + segmentation + tracking)
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 80);
var model = new UNINEXT<double>(architecture, options: new UNINEXTOptions { NumClasses = 80 });
// Or load a pre-trained ONNX model for multi-task instance perception
var onnxModel = new UNINEXT<double>(architecture, "uninext.onnx", options: new UNINEXTOptions { NumClasses = 80 });
Constructors
| Constructor | Summary |
|---|---|
UNINEXT(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, UNINEXTOptions options) |
Initializes UNINEXT in native (trainable) mode. |
UNINEXT(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, UNINEXTOptions options) |
Initializes UNINEXT in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this UNINEXT model’s configuration. |
PanopticSegmentationResult<T> SegmentPanoptic(Tensor<T> image) |