ViTCoMerSegmentationLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

Paper-structured ViT-CoMer encoder and dense-prediction head with parallel ViT/CNN branches, multi-receptive-field refinement, and bidirectional cross-branch interaction.

Inheritance: LayerBase<T> → ViTCoMerSegmentationLayer<T>

Implements: IShapeContract

How It Works

The implementation follows Xia et al., “ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions” (CVPR 2024): a plain ViT remains at 1/16 resolution while a spatial-prior CNN builds a four-level pyramid. Before every ViT stage, 3x3/5x5 depthwise MRFP paths refine the CNN pyramid and cross-attention injects its multi-scale tokens into the ViT. At the stage end a second cross-attention sends the global ViT context back to every CNN scale. A compact FPN-style head consumes all four fused levels.

The paper uses multi-scale deformable attention inside CTI. AiDotNet’s native attention API does not expose learned reference-point offsets, so this layer uses full multi-head cross-attention over the same query/context token sets. It preserves CTI’s directionality, trainable projections, stage placement, and multi-scale information flow while sampling every context location instead of a learned sparse subset.

Constructors

Constructor Summary
ViTCoMerSegmentationLayer(int inputChannels, int inputHeight, int inputWidth, int embedDim, int[] cnnChannels, int[] depths, int decoderDim, int numClasses, double dropRate) Creates the native ViT-CoMer segmentation graph.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank)
void ClearGradients()
void ResetState()
void UpdateParameters(T learningRate)