ConformerBlockLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

A Conformer block — the convolution-augmented transformer block of Gulati et al. 2020 (“Conformer: Convolution-augmented Transformer for Speech Recognition”, arXiv:2005.08100), and the encoder block used by Google USM / Chirp (Zhang et al. 2023, arXiv:2303.01037 §2.1).

Inheritance: LayerBase<T> → ConformerBlockLayer<T>

Implements: IShapeContract

For Beginners

A plain transformer block is good at relating distant parts of a sequence but weak at fine local detail. Speech needs both: phonemes are local, sentences are global. A Conformer block adds a small convolution — which looks at neighbouring time frames — in between the attention and the feed-forward parts, so the model captures local sound patterns and long-range context at once.

How It Works

The block implements the paper’s Equation 1 exactly — a “macaron” sandwich of two HALF-step feed-forward modules around self-attention and a convolution module, every sub-module wrapped in a pre-norm residual, closed by a final LayerNorm:

The convolution module is the part that makes a Conformer a Conformer, and it is what the older Dense-only “conformer” factories in this repository were missing entirely (their own comments admitted the position-wise FFN stood in for a convolution that “was never a real convolution”). Per §2.2 of the paper it is: pointwise convolution with expansion factor 2 → GLU → 1-D DEPTHWISE convolution → normalization → Swish → pointwise convolution.

The convolution module keeps the paper’s BatchNorm. Substituting LayerNorm there — normalizing per time position instead of per channel — is NOT equivalent: its 1/std backward blows up whenever the convolution output at a position is near-degenerate, and it made the very first Adam step produce NaN (parameter L2 34.4 → NaN). BatchNorm normalizes each channel across batch and time, which is well-conditioned for convolution features, and it is what the paper specifies.

One deliberate, documented deviation from the paper:

Input/output layout is [..., time, dim] (time-major, matching the rest of the ASR stack). The convolution module transposes internally to the [batch, channels, time] layout the Conv1D primitives expect and transposes back, so the block is a drop-in for a transformer block. All shape work routes through Engine so the gradient tape records the graph — no manual backward, same contract as the other composite blocks.

Constructors

Constructor Summary
ConformerBlockLayer(int modelDim, int numHeads, int ffnExpansionFactor, int convKernelSize, double ropeTheta, int maxSequenceLength, int attentionGroupSize, int attentionChunkSize, bool causal, int attentionLeftChunks, bool causalConvolution, int feedForwardUnits) Creates a Conformer block.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
void ClearGradients()
void ResetState()
void UpdateParameters(T learningRate)