ConformerBlockLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
A Conformer block — the convolution-augmented transformer block of Gulati et al. 2020 (“Conformer: Convolution-augmented Transformer for Speech Recognition”, arXiv:2005.08100), and the encoder block used by Google USM / Chirp (Zhang et al. 2023, arXiv:2303.01037 §2.1).
Inheritance: LayerBase<T> → ConformerBlockLayer<T>
Implements: IShapeContract
For Beginners
A plain transformer block is good at relating distant parts of a sequence but weak at fine local detail. Speech needs both: phonemes are local, sentences are global. A Conformer block adds a small convolution — which looks at neighbouring time frames — in between the attention and the feed-forward parts, so the model captures local sound patterns and long-range context at once.
How It Works
The block implements the paper’s Equation 1 exactly — a “macaron” sandwich of two HALF-step feed-forward modules around self-attention and a convolution module, every sub-module wrapped in a pre-norm residual, closed by a final LayerNorm:
The convolution module is the part that makes a Conformer a Conformer, and it is what the older Dense-only “conformer” factories in this repository were missing entirely (their own comments admitted the position-wise FFN stood in for a convolution that “was never a real convolution”). Per §2.2 of the paper it is: pointwise convolution with expansion factor 2 → GLU → 1-D DEPTHWISE convolution → normalization → Swish → pointwise convolution.
The convolution module keeps the paper’s BatchNorm. Substituting LayerNorm there — normalizing per time position instead of per channel — is NOT equivalent: its 1/std backward blows up whenever the convolution output at a position is near-degenerate, and it made the very first Adam step produce NaN (parameter L2 34.4 → NaN). BatchNorm normalizes each channel across batch and time, which is well-conditioned for convolution features, and it is what the paper specifies.
One deliberate, documented deviation from the paper:
Input/output layout is [..., time, dim] (time-major, matching the rest of the ASR stack).
The convolution module transposes internally to the [batch, channels, time] layout the
Conv1D primitives expect and transposes back, so the block is a drop-in for a transformer block.
All shape work routes through Engine so the gradient tape records the
graph — no manual backward, same contract as the other composite blocks.
Constructors
| Constructor | Summary |
|---|---|
ConformerBlockLayer(int modelDim, int numHeads, int ffnExpansionFactor, int convKernelSize, double ropeTheta, int maxSequenceLength, int attentionGroupSize, int attentionChunkSize, bool causal, int attentionLeftChunks, bool causalConvolution, int feedForwardUnits) |
Creates a Conformer block. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
void ClearGradients() |
|
void ResetState() |
|
void UpdateParameters(T learningRate) |