LayerCategory
Enums · AiDotNet.Interfaces
Classification of neural network layer types for automated per-layer decisions.
How It Works
For Beginners: Neural networks are made of different types of layers, each with a specific role. This enum categorizes layers so that tools like quantizers, pruners, and pipeline schedulers can make smart per-layer decisions automatically.
For example, a quantizer might keep attention layers at 8-bit precision (they’re sensitive to quantization) while reducing dense layers to 4-bit (they’re more tolerant).
Fields
| Field | Summary |
|---|---|
LayerCategory Activation |
Activation function layers (ReLU, GELU, SiLU, Sigmoid, Tanh, etc.). |
LayerCategory Attention |
Self-attention, cross-attention, multi-head attention layers. |
LayerCategory Capsule |
Capsule network layers (PrimaryCapsule, DigitCapsule, routing). |
LayerCategory Convolution |
Convolutional layers (1D, 2D, 3D, depthwise, separable, dilated, deformable). |
LayerCategory Dense |
Dense/fully-connected layers (e.g., FullyConnectedLayer, DenseLayer). |
LayerCategory Embedding |
Embedding layers (token, positional, patch, time). |
LayerCategory FeedForward |
Feed-forward / MLP block layers. |
LayerCategory Gating |
Gating mechanisms (GLU, SwiGLU, GEGLU). |
LayerCategory Graph |
Graph neural network layers (GCN, GAT, GraphSAGE, GIN, message passing). |
LayerCategory Input |
Input layers. |
LayerCategory Memory |
Memory/external memory layers (NTM read/write heads). |
LayerCategory MixtureOfExperts |
Mixture of experts layers (MoE, Switch, Top-K routing). |
LayerCategory Normalization |
Normalization layers (BatchNorm, LayerNorm, GroupNorm, InstanceNorm, RMSNorm). |
LayerCategory Other |
Custom, specialized, or unclassified layers. |
LayerCategory Pooling |
Pooling layers (MaxPool, AveragePool, AdaptivePool, GlobalPool). |
LayerCategory Positional |
Positional encoding layers (sinusoidal, ALiBi, RoPE, learned). |
LayerCategory Recurrent |
Recurrent layers (LSTM, GRU, RNN, bidirectional). |
LayerCategory Regularization |
Regularization layers (Dropout, GaussianNoise). |
LayerCategory Residual |
Residual/skip connection layers. |
LayerCategory StateSpaceModel |
State Space Model layers (Mamba, S4, S5, RWKV, RetNet). |
LayerCategory Structural |
Reshape, flatten, split, concatenate, and other structural layers. |
LayerCategory Transformer |
Transformer building blocks (EncoderLayer, DecoderLayer). |
LayerCategory Upsampling |
Deconvolution/upsampling layers (TransposedConv, PixelShuffle). |
int value__ |