SegNeXt<T>
Models & Types · AiDotNet.ComputerVision.Segmentation.Semantic
SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation.
Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → SemanticSegmentationBase<T> → SegNeXt<T>
Implements: IGeneratedParameterRegistrar<T>
For Beginners
SegNeXt is a semantic segmentation model that labels every pixel in an image with a category (e.g., road, tree, building). Unlike transformer-based models, SegNeXt uses a purely convolutional backbone called MSCAN (Multi-Scale Convolutional Attention Network) that achieves better accuracy than many transformers while being simpler and faster.
Common use cases:
- Autonomous driving (road, lane, sidewalk parsing)
- Drone/satellite imagery analysis
- Indoor scene understanding
- Real-time segmentation on resource-constrained devices
How It Works
Technical Details:
- MSCAN encoder: a convolutional stem, then blocks of multi-scale convolutional attention (a 5x5 depth-wise
convolution plus 1x7/7x1, 1x11/11x1 and 1x21/21x1 strip branches, mixed by a 1x1 convolution and multiplied into the input) and depth-wise convolutional MLPs, with BatchNorm, layer scale and stochastic depth
- LightHamHead decoder: stages 2-4 at 1/8 scale, concatenated and factored by a non-negative matrix
factorization (the Hamburger) for global context, then a 1x1 classifier; logits are upsampled to the image
- Matches mmsegmentation’s MSCAN and LightHamHead to 1e-9 (SegNeXtReferenceTests) and loads their checkpoints
- Trains with the paper’s recipe: AdamW 6e-5 (head x10, no decay on norms), warm-up then poly decay
Reference: Guo et al., “SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation”, NeurIPS 2022.
Example
// Create a SegNeXt model for convolutional semantic segmentation
var architecture = new NeuralNetworkArchitecture<double>(
inputType: InputType.ThreeDimensional,
taskType: NeuralNetworkTaskType.ImageClassification,
inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new SegNeXt<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SegNeXt<double>(architecture,
options: new SegNeXtOptions { NumClasses = 150, DropRate = 0.1, ModelSize = SegNeXtModelSize.Tiny });
// Or load a pre-trained ONNX model for inference
var onnxModel = new SegNeXt<double>(architecture, "segnext.onnx");
Constructors
| Constructor | Summary |
|---|---|
SegNeXt(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SegNeXtOptions options) |
Creates a trainable SegNeXt with the paper’s architecture for ModelSize. |
SegNeXt(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SegNeXtOptions options) |
Loads an ONNX export for inference. |
Methods
| Method | Summary |
|---|---|
ModelMetadata<T> GetModelMetadata() |
|
Tensor<T> ForwardForTraining(Tensor<T> input) |