ConvSubsamplingLayer<T>

Layers · AiDotNet.NeuralNetworks.Layers

The Conformer’s convolution subsampling front end (Gulati et al. 2020, §2): two 3x3 convolutions with stride 2, each followed by ReLU, over the [time, mel] plane, then a linear projection of the flattened channels and frequencies to the model width. It shortens the sequence four times before the attention blocks.

Inheritance: LayerBase<T> → ConvSubsamplingLayer<T>

Implements: IShapeContract

For Beginners

Speech features arrive 100 times a second. Attention over every one of those frames is slow and unnecessary, so the encoder first learns a convolutional summary of each 40 ms of audio and attends over those instead.

How It Works

The layout follows the reference implementation the paper’s encoder is distributed in (ESPnet’s Conv2dSubsampling): unpadded kernels, so T frames become ((T - 1) / 2 - 1) / 2 and the F mel bins likewise, and the projection takes channels * F' features. Axes before the last two are batch axes and are kept.

Constructors

Constructor Summary
ConvSubsamplingLayer(int inputFeatures, int modelDim) Creates a convolution subsampling front end.

Properties

Property Summary
bool SupportsTraining { get; }

Methods

Method Summary
IReadOnlyList<ILayer<T>> GetSubLayers()
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) Derived from this type’s [TensorLayout] declarations.
int Subsampled(int extent) The length a time or frequency extent has after the two convolutions.
void ResetState()
void UpdateParameters(T learningRate)

Fields

Field Summary
int MinimumExtent The smallest time or frequency extent two unpadded stride-2 3x3 convolutions can reduce.