ConvSubsamplingLayer<T>
Layers · AiDotNet.NeuralNetworks.Layers
The Conformer’s convolution subsampling front end (Gulati et al. 2020, §2): two 3x3 convolutions with stride 2, each followed by ReLU, over the [time, mel] plane, then a linear projection of the flattened channels and frequencies to the model width. It shortens the sequence four times before the attention blocks.
Inheritance: LayerBase<T> → ConvSubsamplingLayer<T>
Implements: IShapeContract
For Beginners
Speech features arrive 100 times a second. Attention over every one of those frames is slow and unnecessary, so the encoder first learns a convolutional summary of each 40 ms of audio and attends over those instead.
How It Works
The layout follows the reference implementation the paper’s encoder is distributed in (ESPnet’s
Conv2dSubsampling): unpadded kernels, so T frames become ((T - 1) / 2 - 1) / 2 and the
F mel bins likewise, and the projection takes channels * F' features. Axes before the last two
are batch axes and are kept.
Constructors
| Constructor | Summary |
|---|---|
ConvSubsamplingLayer(int inputFeatures, int modelDim) |
Creates a convolution subsampling front end. |
Properties
| Property | Summary |
|---|---|
bool SupportsTraining { get; } |
Methods
| Method | Summary |
|---|---|
IReadOnlyList<ILayer<T>> GetSubLayers() |
|
IReadOnlyList<OutputAxisContract> OutputAxesFor(int inputRank) |
Derived from this type’s [TensorLayout] declarations. |
int Subsampled(int extent) |
The length a time or frequency extent has after the two convolutions. |
void ResetState() |
|
void UpdateParameters(T learningRate) |
Fields
| Field | Summary |
|---|---|
int MinimumExtent |
The smallest time or frequency extent two unpadded stride-2 3x3 convolutions can reduce. |