SegFormer<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Semantic
SegFormer: Simple and Efficient Semantic Segmentation with Transformers.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → SemanticSegmentationBase<T> → SegFormer<T>
For Beginners
SegFormer is a semantic segmentation model that classifies every pixel in an image into a category (e.g., road, sky, person, building). It uses a hierarchical transformer encoder (Mix Transformer / MiT) and a lightweight MLP decoder — no complex decoders or positional encodings needed.
Common use cases:
- Autonomous driving (road, lane, obstacle detection)
- Medical imaging (organ/lesion segmentation)
- Scene understanding (indoor/outdoor parsing)
- Agriculture (crop/weed detection from drone images)
How It Works
Technical Details:
- 4-stage hierarchical encoder producing multi-scale features (1/4, 1/8, 1/16, 1/32)
- Overlapping patch embedding replaces non-overlapping ViT patches
- Efficient Self-Attention with spatial reduction for computational savings
- Mix-FFN uses 3x3 depthwise convolutions instead of positional encodings
- Lightweight All-MLP decode head for fast inference
- Six model sizes (B0–B5) from 3.8M to 82.0M parameters
Reference: Xie et al., “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers”, NeurIPS 2021.
Example
// Create a SegFormer model for efficient semantic segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.MultiClassClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new SegFormer<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SegFormer<double>(architecture,
options: new SegFormerOptions { NumClasses = 150, DropRate = 0.1, ModelSize = SegFormerModelSize.B0 });
// Or load a pre-trained ONNX model for autonomous driving
var onnxModel = new SegFormer<double>(architecture, "segformer_b0.onnx");
Constructors
| Constructor | Summary |
|---|---|
SegFormer(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SegFormerOptions options) |
Initializes a new instance of SegFormer in native (trainable) mode. |
SegFormer(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SegFormerOptions options) |
Initializes a new instance of SegFormer in ONNX (inference-only) mode. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
Collects metadata describing this SegFormer model’s configuration and current state. |