LayerCategory

Enums · AiDotNet.Interfaces

Classification of neural network layer types for automated per-layer decisions.

How It Works

For Beginners: Neural networks are made of different types of layers, each with a specific role. This enum categorizes layers so that tools like quantizers, pruners, and pipeline schedulers can make smart per-layer decisions automatically.

For example, a quantizer might keep attention layers at 8-bit precision (they’re sensitive to quantization) while reducing dense layers to 4-bit (they’re more tolerant).

Fields

Field Summary
LayerCategory Activation Activation function layers (ReLU, GELU, SiLU, Sigmoid, Tanh, etc.).
LayerCategory Attention Self-attention, cross-attention, multi-head attention layers.
LayerCategory Capsule Capsule network layers (PrimaryCapsule, DigitCapsule, routing).
LayerCategory Convolution Convolutional layers (1D, 2D, 3D, depthwise, separable, dilated, deformable).
LayerCategory Dense Dense/fully-connected layers (e.g., FullyConnectedLayer, DenseLayer).
LayerCategory Embedding Embedding layers (token, positional, patch, time).
LayerCategory FeedForward Feed-forward / MLP block layers.
LayerCategory Gating Gating mechanisms (GLU, SwiGLU, GEGLU).
LayerCategory Graph Graph neural network layers (GCN, GAT, GraphSAGE, GIN, message passing).
LayerCategory Input Input layers.
LayerCategory Memory Memory/external memory layers (NTM read/write heads).
LayerCategory MixtureOfExperts Mixture of experts layers (MoE, Switch, Top-K routing).
LayerCategory Normalization Normalization layers (BatchNorm, LayerNorm, GroupNorm, InstanceNorm, RMSNorm).
LayerCategory Other Custom, specialized, or unclassified layers.
LayerCategory Pooling Pooling layers (MaxPool, AveragePool, AdaptivePool, GlobalPool).
LayerCategory Positional Positional encoding layers (sinusoidal, ALiBi, RoPE, learned).
LayerCategory Recurrent Recurrent layers (LSTM, GRU, RNN, bidirectional).
LayerCategory Regularization Regularization layers (Dropout, GaussianNoise).
LayerCategory Residual Residual/skip connection layers.
LayerCategory StateSpaceModel State Space Model layers (Mamba, S4, S5, RWKV, RetNet).
LayerCategory Structural Reshape, flatten, split, concatenate, and other structural layers.
LayerCategory Transformer Transformer building blocks (EncoderLayer, DecoderLayer).
LayerCategory Upsampling Deconvolution/upsampling layers (TransposedConv, PixelShuffle).
int value__