SAMHQ<T>

Models & Types · AiDotNet.ComputerVision.Segmentation.Foundation

SAM-HQ: Segment Anything in High Quality.

Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → PromptableSegmentationBase<T> → SAMHQ<T>

For Beginners

SAM-HQ extends Meta’s Segment Anything Model with a High-Quality output token that produces significantly sharper and more accurate mask boundaries. While SAM sometimes produces coarse masks (especially on thin structures like bicycle spokes or fences), SAM-HQ adds learnable components that refine boundaries to pixel-level precision.

Common use cases:

  • High-precision object segmentation where boundary quality matters
  • Thin structure segmentation (wires, fences, poles)
  • Medical imaging requiring precise boundaries
  • Any SAM use case where mask quality needs improvement

How It Works

Technical Details:

  • Adds an HQ output token alongside SAM’s original output tokens
  • Global-local feature fusion: combines early ViT features (local) with final features (global)
  • Trained on only 44K fine-grained masks from HQSeg-44K dataset
  • +17.6 mBIoU improvement on DIS-val5K (thin/complex structures)
  • Backbone: ViT-B/L/H (same as SAM)

Reference: Ke et al., “Segment Anything in High Quality”, NeurIPS 2023.

Example

// Create a SAM-HQ model for high-quality boundary segmentation
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.BinaryClassification,
    inputHeight: 1024, inputWidth: 1024, inputDepth: 3, outputSize: 1);
var model = new SAMHQ<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SAMHQ<double>(architecture,
    options: new SAMHQOptions { NumClasses = 1, DropRate = 0.1, ModelSize = SAMHQModelSize.ViTBase });
            
// Or load a pre-trained ONNX model for precise mask boundaries
var onnxModel = new SAMHQ<double>(architecture, "sam_hq_vit_h.onnx");

Constructors

Constructor Summary
SAMHQ(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SAMHQOptions options) Initializes SAM-HQ in native (trainable) mode.
SAMHQ(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SAMHQOptions options) Initializes SAM-HQ in ONNX (inference-only) mode.

Methods

Method Summary
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
List<PromptedSegmentationResult<T>> SegmentEverything()
ModelMetadata<T> GetModelMetadata() Collects metadata describing this SAM-HQ model’s configuration.
PromptedSegmentationResult<T> SegmentFromBox(Tensor<T> box)
PromptedSegmentationResult<T> SegmentFromMask(Tensor<T> mask)
PromptedSegmentationResult<T> SegmentFromPoints(Tensor<T> points, Tensor<T> labels)
void SetImage(Tensor<T> image)