SegNeXt<T>

Models & Types · AiDotNet.ComputerVision.Segmentation.Semantic

SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation.

Inheritance: NeuralNetworkBase<T> → SegmentationModelBase<T> → SemanticSegmentationBase<T> → SegNeXt<T>

Implements: IGeneratedParameterRegistrar<T>

For Beginners

SegNeXt is a semantic segmentation model that labels every pixel in an image with a category (e.g., road, tree, building). Unlike transformer-based models, SegNeXt uses a purely convolutional backbone called MSCAN (Multi-Scale Convolutional Attention Network) that achieves better accuracy than many transformers while being simpler and faster.

Common use cases:

  • Autonomous driving (road, lane, sidewalk parsing)
  • Drone/satellite imagery analysis
  • Indoor scene understanding
  • Real-time segmentation on resource-constrained devices

How It Works

Technical Details:

  • MSCAN encoder: a convolutional stem, then blocks of multi-scale convolutional attention (a 5x5 depth-wise

convolution plus 1x7/7x1, 1x11/11x1 and 1x21/21x1 strip branches, mixed by a 1x1 convolution and multiplied into the input) and depth-wise convolutional MLPs, with BatchNorm, layer scale and stochastic depth

  • LightHamHead decoder: stages 2-4 at 1/8 scale, concatenated and factored by a non-negative matrix

factorization (the Hamburger) for global context, then a 1x1 classifier; logits are upsampled to the image

  • Matches mmsegmentation’s MSCAN and LightHamHead to 1e-9 (SegNeXtReferenceTests) and loads their checkpoints
  • Trains with the paper’s recipe: AdamW 6e-5 (head x10, no decay on norms), warm-up then poly decay

Reference: Guo et al., “SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentation”, NeurIPS 2022.

Example

// Create a SegNeXt model for convolutional semantic segmentation
var architecture = new NeuralNetworkArchitecture<double>(
    inputType: InputType.ThreeDimensional,
    taskType: NeuralNetworkTaskType.ImageClassification,
    inputHeight: 512, inputWidth: 512, inputDepth: 3, outputSize: 150);
var model = new SegNeXt<double>(architecture);
// Every tunable is now set through the options object; these are the defaults:
var custom = new SegNeXt<double>(architecture,
    options: new SegNeXtOptions { NumClasses = 150, DropRate = 0.1, ModelSize = SegNeXtModelSize.Tiny });
            
// Or load a pre-trained ONNX model for inference
var onnxModel = new SegNeXt<double>(architecture, "segnext.onnx");

Constructors

Constructor Summary
SegNeXt(NeuralNetworkArchitecture<T> architecture, IGradientBasedOptimizer<T, Tensor<T>, Tensor<T>> optimizer, ILossFunction<T> lossFunction, SegNeXtOptions options) Creates a trainable SegNeXt with the paper’s architecture for ModelSize.
SegNeXt(NeuralNetworkArchitecture<T> architecture, string onnxModelPath, SegNeXtOptions options) Loads an ONNX export for inference.

Methods

Method Summary
ModelMetadata<T> GetModelMetadata()
Tensor<T> ForwardForTraining(Tensor<T> input)