SAMHQ<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Foundation
SAM-HQ: Segment Anything in High Quality.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PromptableSegmentationBase<T> → SAMHQ<T>
For Beginners
SAM-HQ extends Meta’s Segment Anything Model with a High-Quality output token that produces significantly sharper and more accurate mask boundaries. While SAM sometimes produces coarse masks (especially on thin structures like bicycle spokes or fences), SAM-HQ adds learnable components that refine boundaries to pixel-level precision.
Common use cases:
- High-precision object segmentation where boundary quality matters
- Thin structure segmentation (wires, fences, poles)
- Medical imaging requiring precise boundaries
- Any SAM use case where mask quality needs improvement
How It Works
Technical Details:
- Adds an HQ output token alongside SAM’s original output tokens
- Global-local feature fusion: combines early ViT features (local) with final features (global)
- Trained on only 44K fine-grained masks from HQSeg-44K dataset
- +17.6 mBIoU improvement on DIS-val5K (thin/complex structures)
- Backbone: ViT-B/L/H (same as SAM)
Reference: Ke et al., “Segment Anything in High Quality”, NeurIPS 2023.
Example
// Create a SAM-HQ model for high-quality boundary segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.BinaryClassification,
inputHeight: 1024, inputWidth: 1024, inputDepth: 3, outputSize: 1);
var model = new SAMHQ<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SAMHQ<double>(architecture,
options: new SAMHQOptions { NumClasses = 1, DropRate = 0.1, ModelSize = SAMHQModelSize.ViTBase });
// Or load a pre-trained ONNX model for precise mask boundaries
var onnxModel = new SAMHQ<double>(architecture, "sam_hq_vit_h.onnx");
Constructors
| Constructor | Summary |
|---|---|
SAMHQ(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SAMHQOptions options) |
Initializes SAM-HQ in native (trainable) mode. |
SAMHQ(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SAMHQOptions options) |
Initializes SAM-HQ in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
List<PromptedSegmentationResult<T>> SegmentEverything() |
|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this SAM-HQ model’s configuration. |
PromptedSegmentationResult<T> SegmentFromBox(Tensor<T> box) |
|
PromptedSegmentationResult<T> SegmentFromMask(Tensor<T> mask) |
|
PromptedSegmentationResult<T> SegmentFromPoints(Tensor<T> points, Tensor<T> labels) |
|
void SetImage(Tensor<T> image) |