ODISESegmentation<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Diffusion
ODISE Segmentation: Panoptic segmentation via diffusion model features.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PanopticSegmentationBase<T> → ODISESegmentation<T>
For Beginners
Diffusion-based panoptic segmentation. Open-vocabulary scene understanding.
Common use cases:
- Diffusion-based panoptic segmentation
- Open-vocabulary scene understanding
- Text-guided segmentation via diffusion
- Novel category segmentation
How It Works
Technical Details:
- Stable Diffusion UNet features for dense prediction
- CLIP text encoder for open-vocabulary category matching
- Mask generator trained on frozen diffusion features
- Joint text-image representation for panoptic labels
Reference: Xu et al., “Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models”, CVPR 2023.
Example
// Create an ODISE model for open-vocabulary panoptic segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new ODISESegmentation<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new ODISESegmentation<double>(architecture,
options: new ODISESegmentationOptions { NumClasses = 133, DropRate = 0.1 });
// Or load a pre-trained ONNX model
var onnxModel = new ODISESegmentation<double>(architecture, "odise_model.onnx");
Constructors
| Constructor | Summary |
|---|---|
ODISESegmentation(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, ODISESegmentationOptions options) |
Initializes ODISESegmentation in native (trainable) mode. |
ODISESegmentation(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ODISESegmentationOptions options) |
Initializes ODISESegmentation in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this model’s configuration. |
PanopticSegmentationResult<T> SegmentPanoptic(Tensor<T> image) |