MaskAdapter<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.OpenVocabulary
Mask-Adapter: Adding SAM to open-vocabulary segmentation via mask prediction.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → OpenVocabSegmentationBase<T> → MaskAdapter<T>
For Beginners
Enhanced open-vocabulary segmentation. SAM-guided mask proposals for open-vocab models.
Common use cases:
- Enhanced open-vocabulary segmentation
- SAM-guided mask proposals for open-vocab models
- Improved boundary quality in zero-shot segmentation
- Plug-and-play mask refinement
How It Works
Technical Details:
- SAM as mask proposal generator for open-vocabulary segmentation
- Mask-level adaptation of CLIP features
- Plug-and-play adapter compatible with multiple backbones
- Improves both boundary quality and category accuracy
Reference: Xie et al., “Mask-Adapter: The Devil is in the Masks for Open-Vocabulary Segmentation”, arXiv 2025.
Example
// Create a MaskAdapter model for SAM-enhanced open-vocabulary segmentation
// Any CLIP-style ITextEncoder supplies the text embeddings, e.g. a CLIP model from VisionLanguage.Encoders.
ITextEncoder<double> textEncoder = clipModel;
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new MaskAdapter<double>(architecture, textEncoder);
// Every tunable is now set through the options object; these are the defaults:
var custom = new MaskAdapter<double>(architecture, textEncoder,
options: new MaskAdapterOptions { NumClasses = 150, DropRate = 0.1 });
// Or load a pre-trained ONNX model for improved boundary quality
var onnxModel = new MaskAdapter<double>(architecture, "maskadapter.onnx", textEncoder, options: null);
Constructors
| Constructor | Summary |
|---|---|
MaskAdapter(NeuralNetworkArchitecture<T> architecture, ITextEncoder<T> textEncoder, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, MaskAdapterOptions options) |
Initializes MaskAdapter in native (trainable) mode. |
MaskAdapter(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, ITextEncoder<T> textEncoder, MaskAdapterOptions options) |
Initializes MaskAdapter in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this model’s configuration. |