ViTCoMerSegmentationLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
Paper-structured ViT-CoMer encoder and dense-prediction head with parallel ViT/CNN branches, multi-receptive-field refinement, and bidirectional cross-branch interaction.
Inheritance: LayerBase<T> → ViTCoMerSegmentationLayer<T>
Implements: IShapeContract
How It Works
The implementation follows Xia et al., “ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions” (CVPR 2024): a plain ViT remains at 1/16 resolution while a spatial-prior CNN builds a four-level pyramid. Before every ViT stage, 3x3/5x5 depthwise MRFP paths refine the CNN pyramid and cross-attention injects its multi-scale tokens into the ViT. At the stage end a second cross-attention sends the global ViT context back to every CNN scale. A compact FPN-style head consumes all four fused levels.
The paper uses multi-scale deformable attention inside CTI. AiDotNet’s native attention API does not expose learned reference-point offsets, so this layer uses full multi-head cross-attention over the same query/context token sets. It preserves CTI’s directionality, trainable projections, stage placement, and multi-scale information flow while sampling every context location instead of a learned sparse subset.
Constructors
| Constructor | Summary |
|---|---|
ViTCoMerSegmentationLayer(int inputChannels, int inputHeight, int inputWidth, int embedDim, int[] cnnChannels, int[] depths, int decoderDim, int numClasses, double dropRate) |
Creates the native ViT-CoMer segmentation graph. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
|
void ClearGradients() |
|
void ResetState() |
|
void UpdateParameters(T learningRate) |